Jul 09 13:53:35.884852 ip-10-0-128-190 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 13:53:35.884866 ip-10-0-128-190 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 13:53:35.884877 ip-10-0-128-190 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 13:53:35.885291 ip-10-0-128-190 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 13:53:45.615247 ip-10-0-128-190 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 09 13:53:45.615264 ip-10-0-128-190 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b62bc83812af4a74b7d1b81f1bb711ff -- Jul 09 13:55:36.027194 ip-10-0-128-190 systemd[1]: Starting Kubernetes Kubelet... Jul 09 13:55:36.437418 ip-10-0-128-190 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 13:55:36.437418 ip-10-0-128-190 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 13:55:36.437418 ip-10-0-128-190 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 13:55:36.437418 ip-10-0-128-190 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 13:55:36.437418 ip-10-0-128-190 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 13:55:36.439018 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.438928 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 13:55:36.444811 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444791 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:55:36.444811 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444810 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444816 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444819 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444822 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444826 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444828 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444831 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444837 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444841 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444857 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444860 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444863 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444866 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444869 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444872 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444882 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444887 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444890 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444894 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:55:36.444901 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444899 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444902 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444905 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444927 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444931 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444934 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444937 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444940 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444943 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444945 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444948 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444950 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444953 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444955 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444958 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444960 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444963 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444965 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444968 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444970 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:55:36.445516 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444972 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444975 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444977 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444980 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444982 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444985 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444987 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444989 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444993 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444995 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.444998 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445000 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445002 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445057 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445273 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445282 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445288 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445293 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445299 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445304 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:55:36.446258 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445309 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445313 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445319 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445323 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445339 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445343 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445353 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445357 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445362 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445366 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445370 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445375 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445379 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445383 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445390 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445394 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445399 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445403 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445412 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445416 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:55:36.446747 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445420 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445424 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445428 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445433 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445436 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445441 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445449 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445453 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445457 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445461 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445466 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445474 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445478 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445482 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445489 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445496 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445500 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445504 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.445508 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:55:36.447249 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446954 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446965 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446969 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446973 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446977 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446980 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446982 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446985 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446987 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446990 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446993 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446996 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.446998 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447001 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447004 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447006 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447010 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447012 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447015 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447018 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:55:36.447710 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447021 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447023 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447028 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447032 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447035 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447037 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447040 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447043 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447045 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447048 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447051 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447053 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447057 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447059 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447062 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447065 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447067 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447070 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447072 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:55:36.448220 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447075 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447077 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447079 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447082 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447084 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447087 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447090 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447092 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447094 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447097 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447099 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447102 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447104 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447107 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447109 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447112 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447114 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447117 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447119 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447121 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:55:36.448688 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447124 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447126 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447128 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447131 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447133 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447137 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447142 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447145 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447148 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447152 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447155 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447157 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447160 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447162 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447165 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447168 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447170 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447172 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447175 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:55:36.449195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447178 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447181 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447183 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447185 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447188 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447191 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447193 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447196 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447198 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447200 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447203 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447205 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447207 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447210 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447212 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447214 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447217 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447220 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447222 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447225 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:55:36.449694 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447228 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447631 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447636 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447638 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447641 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447644 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447647 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447649 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447652 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447654 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447657 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447659 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447662 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447664 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447666 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447669 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447672 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447674 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447677 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447679 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:55:36.450200 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447682 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447685 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447688 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447690 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447692 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447697 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447700 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447703 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447705 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447708 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447711 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447714 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447717 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447721 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447725 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447728 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447730 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447734 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447736 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:55:36.450687 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447739 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447741 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447743 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447746 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447748 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447751 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447753 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447755 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447758 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447760 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447762 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447765 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447767 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447770 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447772 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447774 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447777 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447782 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447785 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447787 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:55:36.451195 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447790 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447793 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447795 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447799 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447801 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447804 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447806 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447809 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447811 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447813 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447816 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447819 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447821 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447824 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447827 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447829 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447832 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447834 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447836 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:55:36.451672 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447839 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447841 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447860 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447863 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447866 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447868 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447871 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447873 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447875 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447878 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447882 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447885 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447887 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447890 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447892 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447894 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447899 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447901 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447904 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:55:36.452146 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447906 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447910 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.447913 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.447986 2575 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.447993 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448001 2575 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448006 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448010 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448013 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448017 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448021 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448024 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448027 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448031 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448034 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448037 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448039 2575 flags.go:64] FLAG: --cgroup-root="" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448042 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448045 2575 flags.go:64] FLAG: --client-ca-file="" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448048 2575 flags.go:64] FLAG: --cloud-provider="external" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448050 2575 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448054 2575 flags.go:64] FLAG: --cluster-domain="" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448057 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 13:55:36.452641 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448061 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448064 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448068 2575 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448072 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448075 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448078 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448083 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448086 2575 flags.go:64] FLAG: --contention-profiling="false" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448089 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448092 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448094 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448097 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448101 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448104 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448107 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448109 2575 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448113 2575 flags.go:64] FLAG: --enable-server="true" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448115 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448121 2575 flags.go:64] FLAG: --event-burst="100" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448124 2575 flags.go:64] FLAG: --event-qps="50" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448126 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448129 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448132 2575 flags.go:64] FLAG: --eviction-hard="" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448136 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 13:55:36.453225 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448139 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448142 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448145 2575 flags.go:64] FLAG: --eviction-soft="" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448148 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448150 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448153 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448156 2575 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448159 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448164 2575 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448166 2575 flags.go:64] FLAG: --feature-gates="" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448170 2575 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448173 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448176 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448179 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448183 2575 flags.go:64] FLAG: --healthz-port="10248" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448186 2575 flags.go:64] FLAG: --help="false" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448189 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448192 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448195 2575 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448197 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448201 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448204 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448206 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 13:55:36.453797 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448209 2575 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448212 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448215 2575 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448218 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448221 2575 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448224 2575 flags.go:64] FLAG: --kube-reserved="" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448227 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448229 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448233 2575 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448235 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448238 2575 flags.go:64] FLAG: --lock-file="" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448241 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448244 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448246 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448251 2575 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448254 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448257 2575 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448261 2575 flags.go:64] FLAG: --logging-format="text" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448264 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448267 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448269 2575 flags.go:64] FLAG: --manifest-url="" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448272 2575 flags.go:64] FLAG: --manifest-url-header="" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448277 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448280 2575 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448284 2575 flags.go:64] FLAG: --max-pods="110" Jul 09 13:55:36.454398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448287 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448289 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448292 2575 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448295 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448298 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448301 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448304 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448310 2575 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448313 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448316 2575 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448320 2575 flags.go:64] FLAG: --pod-cidr="" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448322 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448328 2575 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448330 2575 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448333 2575 flags.go:64] FLAG: --pods-per-core="0" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448336 2575 flags.go:64] FLAG: --port="10250" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448339 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448341 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cf3ffd97cdbc5cdd" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448344 2575 flags.go:64] FLAG: --qos-reserved="" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448347 2575 flags.go:64] FLAG: --read-only-port="10255" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448350 2575 flags.go:64] FLAG: --register-node="true" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448352 2575 flags.go:64] FLAG: --register-with-taints="" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448355 2575 flags.go:64] FLAG: --registry-burst="10" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448358 2575 flags.go:64] FLAG: --registry-qps="5" Jul 09 13:55:36.455014 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448362 2575 flags.go:64] FLAG: --reserved-cpus="" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448365 2575 flags.go:64] FLAG: --reserved-memory="" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448368 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448372 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448374 2575 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448377 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448380 2575 flags.go:64] FLAG: --runonce="false" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448383 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448385 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448388 2575 flags.go:64] FLAG: --seccomp-default="false" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448391 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448394 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448396 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448399 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448402 2575 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448405 2575 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448407 2575 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448410 2575 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448413 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448416 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448419 2575 flags.go:64] FLAG: --system-cgroups="" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448421 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448426 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448428 2575 flags.go:64] FLAG: --tls-cert-file="" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448431 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 13:55:36.455596 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448435 2575 flags.go:64] FLAG: --tls-min-version="" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448438 2575 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448440 2575 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448443 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448445 2575 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448448 2575 flags.go:64] FLAG: --v="2" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448452 2575 flags.go:64] FLAG: --version="false" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448458 2575 flags.go:64] FLAG: --vmodule="" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448461 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.448464 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448587 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448591 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448595 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448598 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448601 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448604 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448608 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448611 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448615 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448617 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448620 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:55:36.456549 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448622 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448625 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448627 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448630 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448632 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448635 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448638 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448640 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448643 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448645 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448647 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448650 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448652 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448655 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448657 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448660 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448662 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448664 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448668 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448670 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:55:36.457474 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448673 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448675 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448678 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448680 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448683 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448686 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448688 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448691 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448694 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448696 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448699 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448701 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448704 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448706 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448710 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448714 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448717 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448719 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448722 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448733 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:55:36.458337 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448736 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448738 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448741 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448743 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448745 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448748 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448750 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448752 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448755 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448757 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448761 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448764 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448766 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448769 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448771 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448773 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448777 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448779 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448782 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448785 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:55:36.459237 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448787 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448790 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448792 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448795 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448797 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448799 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448802 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448806 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448809 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448812 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448814 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448817 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448825 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448828 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448831 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448833 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448835 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448838 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448840 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:55:36.460030 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448843 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448860 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448862 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448866 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448868 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448871 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448873 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448876 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.448886 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.450418 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.458290 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.458315 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.458340 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.458348 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458438 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:55:36.460580 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458447 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458452 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458457 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458462 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458466 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458470 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458475 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458480 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458484 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458491 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458499 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458503 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458508 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458512 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458517 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458521 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458525 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458530 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458534 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458538 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:55:36.461382 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458542 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458547 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458551 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458555 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458559 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458564 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458568 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458573 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458578 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458583 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458587 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458592 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458596 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458600 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458604 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458608 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458612 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458617 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458621 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458625 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:55:36.462239 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458629 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458633 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458637 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458643 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458647 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458651 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458655 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458660 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458664 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458668 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458672 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458677 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458681 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458686 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458690 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458694 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458698 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458702 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458707 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458711 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:55:36.462883 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458716 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458720 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458727 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458732 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458737 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458741 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458746 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458751 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458755 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458759 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458764 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458770 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458775 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458780 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458785 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458789 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458798 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458802 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458807 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:55:36.463429 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458812 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458816 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458819 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458823 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458827 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458832 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458836 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458839 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458844 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458872 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458876 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458880 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458885 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458889 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458894 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458898 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458903 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458907 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:55:36.463926 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.458910 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.458919 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459141 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459149 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459154 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459159 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459163 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459168 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459172 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459176 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459180 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459184 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459189 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459193 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459197 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 13:55:36.464432 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459201 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459205 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459210 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459214 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459218 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459222 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459226 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459231 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459235 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459240 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459243 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459247 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459251 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459255 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459259 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459263 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459267 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459271 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459275 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459279 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 13:55:36.464805 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459284 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459288 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459292 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459296 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459300 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459304 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459308 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459312 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459316 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459320 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459325 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459330 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459334 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459338 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459342 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459346 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459350 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459354 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459358 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459363 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 13:55:36.465388 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459367 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459371 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459375 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459379 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459383 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459387 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459392 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459396 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459399 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459403 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459408 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459412 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459416 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459420 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459425 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459429 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459433 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459439 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459445 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 13:55:36.465952 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459449 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459454 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459458 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459463 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459470 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459475 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459480 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459485 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459489 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459493 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459497 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459502 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459507 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459511 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459515 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459519 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459523 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459527 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459532 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 13:55:36.466498 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459537 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459541 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459545 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459549 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459554 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459558 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459564 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:36.459569 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.459577 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.459761 2575 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.462637 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.464070 2575 server.go:1019] "Starting client certificate rotation" Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.464190 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 13:55:36.467310 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.464230 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 13:55:36.467891 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.467109 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 13:55:36.470805 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.470789 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 13:55:36.481473 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.481438 2575 log.go:25] "Validated CRI v1 runtime API" Jul 09 13:55:36.484489 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.484470 2575 log.go:25] "Validated CRI v1 image API" Jul 09 13:55:36.485361 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.485345 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 13:55:36.488331 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.488310 2575 fs.go:135] Filesystem UUIDs: map[50f00941-0a26-4ada-8af4-b7570713379f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b90f60c6-fffb-417e-8a2f-e4db033072ea:/dev/nvme0n1p3] Jul 09 13:55:36.488394 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.488332 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 13:55:36.493514 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.493388 2575 manager.go:216] Machine: {Timestamp:2026-07-09 13:55:36.49219701 +0000 UTC m=+0.346148477 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104145 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28409dafb40feda70c7d68eef5a740 SystemUUID:ec28409d-afb4-0fed-a70c-7d68eef5a740 BootID:b62bc838-12af-4a74-b7d1-b81f1bb711ff Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d9:09:66:03:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d9:09:66:03:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:f2:9a:96:df:08 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 13:55:36.493514 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.493510 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 13:55:36.493636 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.493601 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 13:55:36.497038 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.497000 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 13:55:36.497191 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.497037 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-190.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 13:55:36.497782 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.497769 2575 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 13:55:36.497782 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.497784 2575 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 13:55:36.497915 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.497792 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 13:55:36.498436 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.498424 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 13:55:36.499795 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.499784 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 13:55:36.500026 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.500009 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 13:55:36.502956 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.502940 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 09 13:55:36.503089 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.503079 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 13:55:36.503141 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.503131 2575 kubelet.go:486] "Attempting to sync node with API server" Jul 09 13:55:36.503170 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.503149 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 13:55:36.503170 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.503161 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 13:55:36.503170 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.503171 2575 kubelet.go:398] "Adding apiserver pod source" Jul 09 13:55:36.503273 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.503180 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 13:55:36.503604 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.503589 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 13:55:36.503695 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.503611 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 13:55:36.510748 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.510729 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 13:55:36.512787 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.512768 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 13:55:36.512833 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.512806 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 13:55:36.513971 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.513944 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 13:55:36.514109 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.514089 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-190.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 13:55:36.514864 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514835 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 13:55:36.514930 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514871 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 13:55:36.514930 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514883 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 13:55:36.514930 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514890 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 13:55:36.514930 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514899 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 13:55:36.514930 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514904 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 13:55:36.514930 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514912 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 13:55:36.514930 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514917 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 13:55:36.514930 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514925 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 13:55:36.515141 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514937 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 13:55:36.515141 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514950 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 13:55:36.515141 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.514955 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 13:55:36.515662 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.515642 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 13:55:36.515662 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.515657 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 13:55:36.518961 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.518947 2575 server.go:1268] "Started kubelet" Jul 09 13:55:36.519089 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.519039 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 13:55:36.519153 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.519048 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 13:55:36.519211 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.519183 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 13:55:36.519882 ip-10-0-128-190 systemd[1]: Started Kubernetes Kubelet. Jul 09 13:55:36.521241 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.521222 2575 server.go:310] "Adding debug handlers to kubelet server" Jul 09 13:55:36.521870 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.521826 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 13:55:36.524983 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.524948 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 13:55:36.527502 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.527485 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-190.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 13:55:36.528358 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.527521 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-190.ec2.internal.18c0a39a86b380b9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-190.ec2.internal,UID:ip-10-0-128-190.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-190.ec2.internal,},FirstTimestamp:2026-07-09 13:55:36.518918329 +0000 UTC m=+0.372869796,LastTimestamp:2026-07-09 13:55:36.518918329 +0000 UTC m=+0.372869796,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-190.ec2.internal,}" Jul 09 13:55:36.532495 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.532468 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 13:55:36.532619 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.532493 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 13:55:36.532619 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.532610 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:36.533179 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533164 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 13:55:36.533267 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533167 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 13:55:36.533267 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533207 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 13:55:36.533398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533318 2575 reconstruct.go:97] "Volume reconstruction finished" Jul 09 13:55:36.533398 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533327 2575 reconciler.go:29] "Reconciler: start to sync state" Jul 09 13:55:36.533569 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533552 2575 factory.go:55] Registering systemd factory Jul 09 13:55:36.533569 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533571 2575 factory.go:223] Registration of the systemd container factory successfully Jul 09 13:55:36.533840 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533821 2575 factory.go:153] Registering CRI-O factory Jul 09 13:55:36.533840 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533842 2575 factory.go:223] Registration of the crio container factory successfully Jul 09 13:55:36.534011 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533955 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 13:55:36.534011 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533982 2575 factory.go:103] Registering Raw factory Jul 09 13:55:36.534011 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.533998 2575 manager.go:1196] Started watching for new ooms in manager Jul 09 13:55:36.534453 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.534440 2575 manager.go:349] Starting recovery of all containers Jul 09 13:55:36.535038 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.535010 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-190.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 13:55:36.535124 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.535106 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 13:55:36.545843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.545710 2575 manager.go:354] Recovery completed Jul 09 13:55:36.550024 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.550004 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:55:36.551271 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.551248 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vw425" Jul 09 13:55:36.552414 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.552391 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:55:36.552489 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.552436 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:55:36.552489 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.552452 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:55:36.553113 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.553096 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 13:55:36.553113 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.553111 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 13:55:36.553243 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.553114 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:55:36.553243 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.553131 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 09 13:55:36.553918 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.553901 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:55:36.553990 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.553931 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:55:36.553990 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.553941 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:55:36.554430 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.554418 2575 policy_none.go:49] "None policy: Start" Jul 09 13:55:36.554495 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.554434 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 13:55:36.554495 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.554445 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 13:55:36.555326 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.555315 2575 policy_none.go:47] "Start" Jul 09 13:55:36.560437 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.560420 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vw425" Jul 09 13:55:36.600667 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.600634 2575 manager.go:337] "Starting Device Plugin manager" Jul 09 13:55:36.600829 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.600687 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 13:55:36.600829 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.600702 2575 server.go:87] "Starting device plugin registration server" Jul 09 13:55:36.601068 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.601053 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 13:55:36.601148 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.601073 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 13:55:36.601192 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.601148 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 13:55:36.601266 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.601252 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 13:55:36.601266 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.601266 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 13:55:36.601821 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.601797 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 13:55:36.601940 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.601882 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:36.629569 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.629533 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 13:55:36.631035 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.631013 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 13:55:36.631135 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.631058 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 13:55:36.631135 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.631084 2575 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 13:55:36.631135 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.631126 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 13:55:36.633616 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.633585 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:55:36.701749 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.701641 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:55:36.702772 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.702755 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:55:36.702833 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.702795 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:55:36.702833 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.702811 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:55:36.702961 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.702836 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.712731 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.712706 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.712821 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.712736 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-190.ec2.internal\": node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:36.731562 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.731529 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal"] Jul 09 13:55:36.731713 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.731604 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:55:36.732589 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.732572 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:55:36.732697 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.732599 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:55:36.732697 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.732608 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:55:36.733949 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.733935 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:55:36.734076 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.734053 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.734132 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.734091 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:55:36.734671 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.734654 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:55:36.734741 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.734686 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:55:36.734741 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.734697 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:55:36.734741 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.734714 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:55:36.734741 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.734734 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:55:36.734741 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.734743 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:55:36.735977 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.735960 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.736085 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.735987 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 13:55:36.736634 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.736612 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientMemory" Jul 09 13:55:36.736724 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.736647 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 13:55:36.736724 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.736662 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeHasSufficientPID" Jul 09 13:55:36.757116 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.757087 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:36.760694 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.760674 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-190.ec2.internal\" not found" node="ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.765210 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.765192 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-190.ec2.internal\" not found" node="ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.834213 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.834180 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e2ef5c33e6feccabcbd54c585584e1d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal\" (UID: \"1e2ef5c33e6feccabcbd54c585584e1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.834370 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.834224 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e2ef5c33e6feccabcbd54c585584e1d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal\" (UID: \"1e2ef5c33e6feccabcbd54c585584e1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.857394 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.857364 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:36.934810 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.934778 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e2ef5c33e6feccabcbd54c585584e1d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal\" (UID: \"1e2ef5c33e6feccabcbd54c585584e1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.934927 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.934813 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e2ef5c33e6feccabcbd54c585584e1d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal\" (UID: \"1e2ef5c33e6feccabcbd54c585584e1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.934927 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.934838 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3de1e4a975b796018b205e668ce5af6-config\") pod \"kube-apiserver-proxy-ip-10-0-128-190.ec2.internal\" (UID: \"b3de1e4a975b796018b205e668ce5af6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.934927 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.934912 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e2ef5c33e6feccabcbd54c585584e1d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal\" (UID: \"1e2ef5c33e6feccabcbd54c585584e1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.935036 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:36.934913 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e2ef5c33e6feccabcbd54c585584e1d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal\" (UID: \"1e2ef5c33e6feccabcbd54c585584e1d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" Jul 09 13:55:36.957964 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:36.957896 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:37.035648 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.035607 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3de1e4a975b796018b205e668ce5af6-config\") pod \"kube-apiserver-proxy-ip-10-0-128-190.ec2.internal\" (UID: \"b3de1e4a975b796018b205e668ce5af6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal" Jul 09 13:55:37.035745 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.035680 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3de1e4a975b796018b205e668ce5af6-config\") pod \"kube-apiserver-proxy-ip-10-0-128-190.ec2.internal\" (UID: \"b3de1e4a975b796018b205e668ce5af6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal" Jul 09 13:55:37.058771 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:37.058739 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:37.064927 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.064902 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" Jul 09 13:55:37.069377 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.069292 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal" Jul 09 13:55:37.159602 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:37.159562 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:37.260175 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:37.260097 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:37.360624 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:37.360592 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:37.461010 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:37.460965 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:37.463254 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.463231 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 13:55:37.463445 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.463418 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 13:55:37.550869 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.550765 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:55:37.562115 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:37.562082 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:37.563277 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.563247 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 13:50:36 +0000 UTC" deadline="2027-12-07 22:09:32.641194727 +0000 UTC" Jul 09 13:55:37.563327 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.563278 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12392h13m55.077920125s" Jul 09 13:55:37.663615 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:37.663564 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-190.ec2.internal\" not found" Jul 09 13:55:37.742535 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.742512 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:55:37.837542 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.837452 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" Jul 09 13:55:37.852517 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.852485 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 13:55:37.853945 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.853910 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal" Jul 09 13:55:37.861666 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:37.861646 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 13:55:38.135421 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.135350 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:55:38.299525 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.299482 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e2ef5c33e6feccabcbd54c585584e1d.slice/crio-6c39ec8622c6b91625b7e5609cbdb914ff1445f1e218f5f4e2c89123966056c7 WatchSource:0}: Error finding container 6c39ec8622c6b91625b7e5609cbdb914ff1445f1e218f5f4e2c89123966056c7: Status 404 returned error can't find the container with id 6c39ec8622c6b91625b7e5609cbdb914ff1445f1e218f5f4e2c89123966056c7 Jul 09 13:55:38.303678 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.303660 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 13:55:38.505921 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.505799 2575 apiserver.go:52] "Watching apiserver" Jul 09 13:55:38.516741 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.515885 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 13:55:38.517480 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.517451 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-t2rdv","openshift-network-operator/iptables-alerter-wzgmj","kube-system/global-pull-secret-syncer-k4zsb","openshift-image-registry/node-ca-5tgvd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal","openshift-multus/multus-m2hqg","openshift-network-diagnostics/network-check-target-slr49","openshift-ovn-kubernetes/ovnkube-node-h84mk","kube-system/konnectivity-agent-7f7k4","kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm","openshift-cluster-node-tuning-operator/tuned-6nkn4","openshift-dns/node-resolver-xd872","openshift-multus/multus-additional-cni-plugins-zs2sz"] Jul 09 13:55:38.519898 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.519873 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:38.520020 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.519958 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:38.521084 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.521064 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.522452 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.522433 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:38.522560 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.522501 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:38.523661 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.523633 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.523831 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.523810 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 13:55:38.523960 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.523822 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 13:55:38.524034 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.523967 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 13:55:38.524097 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.524078 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-668gb\"" Jul 09 13:55:38.525113 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.525094 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.526094 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.526040 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 13:55:38.526302 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.526203 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 13:55:38.526400 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.526381 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.526482 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.526466 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 13:55:38.527130 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.527111 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kft7q\"" Jul 09 13:55:38.527483 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.527460 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 13:55:38.527649 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.527631 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:55:38.529030 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.529012 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.529763 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.529738 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hhq9q\"" Jul 09 13:55:38.529763 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.529762 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 13:55:38.529985 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.529747 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 13:55:38.530149 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.530129 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 13:55:38.530271 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.530221 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 13:55:38.530271 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.530266 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 13:55:38.530432 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.530393 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cfmng\"" Jul 09 13:55:38.530515 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.530499 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kcnns\"" Jul 09 13:55:38.530584 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.530561 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.531362 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.531346 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 13:55:38.531462 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.531347 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 13:55:38.531462 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.531419 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 13:55:38.531462 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.531420 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w2fbl\"" Jul 09 13:55:38.531462 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.531352 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 13:55:38.532038 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.532020 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.533586 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.533566 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:38.533691 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.533665 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:38.533759 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.533701 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 13:55:38.535507 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.535434 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 13:55:38.535507 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.535437 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 13:55:38.536174 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.536090 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 13:55:38.536174 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.536102 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 13:55:38.536311 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.536296 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 13:55:38.536375 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.536359 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-b5mht\"" Jul 09 13:55:38.536454 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.536436 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 13:55:38.536667 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.536650 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wsnjp\"" Jul 09 13:55:38.536739 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.536719 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 13:55:38.536779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.536741 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 13:55:38.536829 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.536816 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.537403 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.537386 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 13:55:38.539408 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.539387 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9m2cr\"" Jul 09 13:55:38.539496 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.539417 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 13:55:38.539643 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.539624 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 13:55:38.543160 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543137 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-log-socket\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.543257 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543177 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-run\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.543351 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543329 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-tmp\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.543454 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543360 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/412a8a2b-3312-4a3f-be35-dbba3ef982eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.543454 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543391 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-slash\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.543454 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543424 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-etc-openvswitch\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.543590 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543463 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/232c0b60-887d-41bd-8563-67bd303f57e4-host\") pod \"node-ca-5tgvd\" (UID: \"232c0b60-887d-41bd-8563-67bd303f57e4\") " pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.543590 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543481 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzc2q\" (UniqueName: \"kubernetes.io/projected/232c0b60-887d-41bd-8563-67bd303f57e4-kube-api-access-zzc2q\") pod \"node-ca-5tgvd\" (UID: \"232c0b60-887d-41bd-8563-67bd303f57e4\") " pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.543590 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543506 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-run-netns\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.543590 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543541 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-cni-netd\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.543590 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543568 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:38.543779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543596 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-sysctl-conf\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.543779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543619 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-host\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.543779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543635 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpskm\" (UniqueName: \"kubernetes.io/projected/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-kube-api-access-tpskm\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.543779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543651 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-node-log\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.543779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543667 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-conf-dir\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.543779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543699 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr2fh\" (UniqueName: \"kubernetes.io/projected/4e1ac675-9682-4468-993d-3d5183940a54-kube-api-access-xr2fh\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:38.543779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543754 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9137871a-fadb-43ef-acb4-c5917ed17a72-dbus\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:38.543779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543778 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a875e964-90d0-43e9-8da1-c427093ead9e-agent-certs\") pod \"konnectivity-agent-7f7k4\" (UID: \"a875e964-90d0-43e9-8da1-c427093ead9e\") " pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543797 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45e4c2e5-2a9c-4508-a9d1-e2c274713331-tmp-dir\") pod \"node-resolver-xd872\" (UID: \"45e4c2e5-2a9c-4508-a9d1-e2c274713331\") " pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543813 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgxgp\" (UniqueName: \"kubernetes.io/projected/45e4c2e5-2a9c-4508-a9d1-e2c274713331-kube-api-access-tgxgp\") pod \"node-resolver-xd872\" (UID: \"45e4c2e5-2a9c-4508-a9d1-e2c274713331\") " pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543828 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64048b69-f5b1-409d-a9ec-06ffab09b71f-ovnkube-config\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543843 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64048b69-f5b1-409d-a9ec-06ffab09b71f-env-overrides\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543876 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-var-lib-cni-bin\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543902 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-var-lib-kubelet\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543923 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-modprobe-d\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543938 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s69jm\" (UniqueName: \"kubernetes.io/projected/412a8a2b-3312-4a3f-be35-dbba3ef982eb-kube-api-access-s69jm\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543964 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-run-ovn-kubernetes\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543979 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cngl\" (UniqueName: \"kubernetes.io/projected/399ce845-2671-457b-a670-7ee180cc1d67-kube-api-access-7cngl\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.543995 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-os-release\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544010 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-lib-modules\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544027 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/232c0b60-887d-41bd-8563-67bd303f57e4-serviceca\") pod \"node-ca-5tgvd\" (UID: \"232c0b60-887d-41bd-8563-67bd303f57e4\") " pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544043 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/412a8a2b-3312-4a3f-be35-dbba3ef982eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544067 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-run-openvswitch\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.544098 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544088 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64048b69-f5b1-409d-a9ec-06ffab09b71f-ovn-node-metrics-cert\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544106 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-socket-dir-parent\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544122 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-var-lib-cni-multus\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544161 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lncmg\" (UniqueName: \"kubernetes.io/projected/36f61295-2489-44e4-b59d-51d77bd5f40b-kube-api-access-lncmg\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544195 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-cnibin\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544221 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-os-release\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544247 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/412a8a2b-3312-4a3f-be35-dbba3ef982eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544274 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b65e29e7-0841-46af-9714-e71a7444329d-iptables-alerter-script\") pod \"iptables-alerter-wzgmj\" (UID: \"b65e29e7-0841-46af-9714-e71a7444329d\") " pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544302 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmf2\" (UniqueName: \"kubernetes.io/projected/b65e29e7-0841-46af-9714-e71a7444329d-kube-api-access-cbmf2\") pod \"iptables-alerter-wzgmj\" (UID: \"b65e29e7-0841-46af-9714-e71a7444329d\") " pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544329 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-kubelet\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544355 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-etc-kubernetes\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544385 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-daemon-config\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-var-lib-kubelet\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-system-cni-dir\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544462 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544487 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-system-cni-dir\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.544732 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544512 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-cni-dir\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544535 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-hostroot\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544564 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b65e29e7-0841-46af-9714-e71a7444329d-host-slash\") pod \"iptables-alerter-wzgmj\" (UID: \"b65e29e7-0841-46af-9714-e71a7444329d\") " pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544599 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544640 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-etc-selinux\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544686 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-run-netns\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544720 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-sysconfig\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544742 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-sys\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544767 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-tuned\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544792 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/45e4c2e5-2a9c-4508-a9d1-e2c274713331-hosts-file\") pod \"node-resolver-xd872\" (UID: \"45e4c2e5-2a9c-4508-a9d1-e2c274713331\") " pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544817 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-socket-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544843 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-device-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544868 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-run-k8s-cni-cncf-io\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544924 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-run-multus-certs\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544951 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-kubernetes\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.544982 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-systemd\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.545504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545004 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9137871a-fadb-43ef-acb4-c5917ed17a72-kubelet-config\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545021 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545037 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-run-systemd\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545052 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-run-ovn\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545066 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-cni-bin\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545095 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-cnibin\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545122 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36f61295-2489-44e4-b59d-51d77bd5f40b-cni-binary-copy\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545153 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545180 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64048b69-f5b1-409d-a9ec-06ffab09b71f-ovnkube-script-lib\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545215 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-sysctl-d\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545243 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-systemd-units\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545271 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-var-lib-openvswitch\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545295 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-sys-fs\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545322 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545347 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a875e964-90d0-43e9-8da1-c427093ead9e-konnectivity-ca\") pod \"konnectivity-agent-7f7k4\" (UID: \"a875e964-90d0-43e9-8da1-c427093ead9e\") " pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545372 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shnfd\" (UniqueName: \"kubernetes.io/projected/64048b69-f5b1-409d-a9ec-06ffab09b71f-kube-api-access-shnfd\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.546067 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.545428 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-registration-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.550106 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.549995 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 13:55:38.578579 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.578559 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pxcbc" Jul 09 13:55:38.590773 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.590734 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pxcbc" Jul 09 13:55:38.633869 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.633838 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 13:55:38.636222 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.636171 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" event={"ID":"1e2ef5c33e6feccabcbd54c585584e1d","Type":"ContainerStarted","Data":"6c39ec8622c6b91625b7e5609cbdb914ff1445f1e218f5f4e2c89123966056c7"} Jul 09 13:55:38.637497 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.637476 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal" event={"ID":"b3de1e4a975b796018b205e668ce5af6","Type":"ContainerStarted","Data":"d36e43873158148e5a311e46c1d8067e6801fcb2831b5320ad0df6f39c9f0f01"} Jul 09 13:55:38.646412 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646388 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a875e964-90d0-43e9-8da1-c427093ead9e-agent-certs\") pod \"konnectivity-agent-7f7k4\" (UID: \"a875e964-90d0-43e9-8da1-c427093ead9e\") " pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:55:38.646527 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646420 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45e4c2e5-2a9c-4508-a9d1-e2c274713331-tmp-dir\") pod \"node-resolver-xd872\" (UID: \"45e4c2e5-2a9c-4508-a9d1-e2c274713331\") " pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.646527 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646437 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgxgp\" (UniqueName: \"kubernetes.io/projected/45e4c2e5-2a9c-4508-a9d1-e2c274713331-kube-api-access-tgxgp\") pod \"node-resolver-xd872\" (UID: \"45e4c2e5-2a9c-4508-a9d1-e2c274713331\") " pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.646527 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646454 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64048b69-f5b1-409d-a9ec-06ffab09b71f-ovnkube-config\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.646689 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646526 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64048b69-f5b1-409d-a9ec-06ffab09b71f-env-overrides\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.646689 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646562 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-var-lib-cni-bin\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.646689 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646591 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-var-lib-kubelet\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.646856 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646687 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-var-lib-kubelet\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.646856 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646723 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-var-lib-cni-bin\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.646856 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646813 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-modprobe-d\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.646856 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646831 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45e4c2e5-2a9c-4508-a9d1-e2c274713331-tmp-dir\") pod \"node-resolver-xd872\" (UID: \"45e4c2e5-2a9c-4508-a9d1-e2c274713331\") " pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.647066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646849 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s69jm\" (UniqueName: \"kubernetes.io/projected/412a8a2b-3312-4a3f-be35-dbba3ef982eb-kube-api-access-s69jm\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.647066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646880 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 13:55:38.647066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646897 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-run-ovn-kubernetes\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.647066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646924 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cngl\" (UniqueName: \"kubernetes.io/projected/399ce845-2671-457b-a670-7ee180cc1d67-kube-api-access-7cngl\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.647066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-os-release\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.647066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646968 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-modprobe-d\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.647066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646973 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-lib-modules\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.647066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.646982 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-run-ovn-kubernetes\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.647066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647043 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/232c0b60-887d-41bd-8563-67bd303f57e4-serviceca\") pod \"node-ca-5tgvd\" (UID: \"232c0b60-887d-41bd-8563-67bd303f57e4\") " pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.647415 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647069 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64048b69-f5b1-409d-a9ec-06ffab09b71f-env-overrides\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.647415 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647094 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-os-release\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.647415 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647140 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-lib-modules\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.647415 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647154 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64048b69-f5b1-409d-a9ec-06ffab09b71f-ovnkube-config\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.647415 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/412a8a2b-3312-4a3f-be35-dbba3ef982eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.647415 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647282 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-run-openvswitch\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.647415 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647325 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64048b69-f5b1-409d-a9ec-06ffab09b71f-ovn-node-metrics-cert\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.647415 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647380 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-run-openvswitch\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.647415 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647414 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-socket-dir-parent\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-var-lib-cni-multus\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647478 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-socket-dir-parent\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647497 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lncmg\" (UniqueName: \"kubernetes.io/projected/36f61295-2489-44e4-b59d-51d77bd5f40b-kube-api-access-lncmg\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647525 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-cnibin\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647555 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-os-release\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647562 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-var-lib-cni-multus\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647582 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/412a8a2b-3312-4a3f-be35-dbba3ef982eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647599 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/232c0b60-887d-41bd-8563-67bd303f57e4-serviceca\") pod \"node-ca-5tgvd\" (UID: \"232c0b60-887d-41bd-8563-67bd303f57e4\") " pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647612 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-cnibin\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647614 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b65e29e7-0841-46af-9714-e71a7444329d-iptables-alerter-script\") pod \"iptables-alerter-wzgmj\" (UID: \"b65e29e7-0841-46af-9714-e71a7444329d\") " pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647654 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmf2\" (UniqueName: \"kubernetes.io/projected/b65e29e7-0841-46af-9714-e71a7444329d-kube-api-access-cbmf2\") pod \"iptables-alerter-wzgmj\" (UID: \"b65e29e7-0841-46af-9714-e71a7444329d\") " pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647681 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-kubelet\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647706 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-etc-kubernetes\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647722 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-os-release\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647732 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-daemon-config\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647758 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-var-lib-kubelet\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.647843 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647812 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-etc-kubernetes\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647824 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/412a8a2b-3312-4a3f-be35-dbba3ef982eb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647842 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-system-cni-dir\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647869 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647912 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-system-cni-dir\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647965 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-cni-dir\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.647995 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-hostroot\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b65e29e7-0841-46af-9714-e71a7444329d-host-slash\") pod \"iptables-alerter-wzgmj\" (UID: \"b65e29e7-0841-46af-9714-e71a7444329d\") " pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648107 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-var-lib-kubelet\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648134 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-etc-selinux\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648150 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/412a8a2b-3312-4a3f-be35-dbba3ef982eb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648201 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-run-netns\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648109 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b65e29e7-0841-46af-9714-e71a7444329d-host-slash\") pod \"iptables-alerter-wzgmj\" (UID: \"b65e29e7-0841-46af-9714-e71a7444329d\") " pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648237 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-sysconfig\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648273 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-sys\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648277 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-run-netns\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.648723 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648302 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-tuned\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648342 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-daemon-config\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648356 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-sysconfig\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648204 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648372 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-etc-selinux\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648390 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/45e4c2e5-2a9c-4508-a9d1-e2c274713331-hosts-file\") pod \"node-resolver-xd872\" (UID: \"45e4c2e5-2a9c-4508-a9d1-e2c274713331\") " pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648421 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-socket-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648423 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-sys\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-device-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648456 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-system-cni-dir\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648494 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/45e4c2e5-2a9c-4508-a9d1-e2c274713331-hosts-file\") pod \"node-resolver-xd872\" (UID: \"45e4c2e5-2a9c-4508-a9d1-e2c274713331\") " pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648503 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-device-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648511 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-run-k8s-cni-cncf-io\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648542 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-kubelet\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648569 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-run-multus-certs\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648585 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-hostroot\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648593 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-socket-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648604 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-kubernetes\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.649540 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648641 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-cni-dir\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648650 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648653 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-run-k8s-cni-cncf-io\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648680 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-system-cni-dir\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648693 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b65e29e7-0841-46af-9714-e71a7444329d-iptables-alerter-script\") pod \"iptables-alerter-wzgmj\" (UID: \"b65e29e7-0841-46af-9714-e71a7444329d\") " pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648710 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-host-run-multus-certs\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648711 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-systemd\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648698 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-kubernetes\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9137871a-fadb-43ef-acb4-c5917ed17a72-kubelet-config\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648821 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-systemd\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648834 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.648866 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9137871a-fadb-43ef-acb4-c5917ed17a72-kubelet-config\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649041 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/412a8a2b-3312-4a3f-be35-dbba3ef982eb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649071 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-run-systemd\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649164 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-run-systemd\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649172 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-run-ovn\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649203 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-cni-bin\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.650461 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649234 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-cnibin\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649240 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-run-ovn\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649264 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36f61295-2489-44e4-b59d-51d77bd5f40b-cni-binary-copy\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649295 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649305 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-cni-bin\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649328 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64048b69-f5b1-409d-a9ec-06ffab09b71f-ovnkube-script-lib\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649360 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-sysctl-d\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649388 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-systemd-units\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649390 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-cnibin\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649419 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-var-lib-openvswitch\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-sys-fs\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649480 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649515 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a875e964-90d0-43e9-8da1-c427093ead9e-konnectivity-ca\") pod \"konnectivity-agent-7f7k4\" (UID: \"a875e964-90d0-43e9-8da1-c427093ead9e\") " pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shnfd\" (UniqueName: \"kubernetes.io/projected/64048b69-f5b1-409d-a9ec-06ffab09b71f-kube-api-access-shnfd\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649607 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-registration-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649655 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-log-socket\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649700 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-run\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.651333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649728 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-tmp\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649758 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/412a8a2b-3312-4a3f-be35-dbba3ef982eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649786 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-slash\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649813 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-etc-openvswitch\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649840 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/232c0b60-887d-41bd-8563-67bd303f57e4-host\") pod \"node-ca-5tgvd\" (UID: \"232c0b60-887d-41bd-8563-67bd303f57e4\") " pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649868 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzc2q\" (UniqueName: \"kubernetes.io/projected/232c0b60-887d-41bd-8563-67bd303f57e4-kube-api-access-zzc2q\") pod \"node-ca-5tgvd\" (UID: \"232c0b60-887d-41bd-8563-67bd303f57e4\") " pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649895 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-run-netns\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649886 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36f61295-2489-44e4-b59d-51d77bd5f40b-cni-binary-copy\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-cni-netd\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.649979 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650007 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-sysctl-conf\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.650020 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650033 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-host\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650062 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpskm\" (UniqueName: \"kubernetes.io/projected/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-kube-api-access-tpskm\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.650114 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:39.150079233 +0000 UTC m=+3.004030710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650139 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-node-log\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-conf-dir\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.651988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650202 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr2fh\" (UniqueName: \"kubernetes.io/projected/4e1ac675-9682-4468-993d-3d5183940a54-kube-api-access-xr2fh\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650229 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9137871a-fadb-43ef-acb4-c5917ed17a72-dbus\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650373 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9137871a-fadb-43ef-acb4-c5917ed17a72-dbus\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650411 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-sysctl-d\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650470 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-systemd-units\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650545 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-var-lib-openvswitch\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650605 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-sys-fs\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.650701 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.650751 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret podName:9137871a-fadb-43ef-acb4-c5917ed17a72 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:39.150734793 +0000 UTC m=+3.004686265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret") pod "global-pull-secret-syncer-k4zsb" (UID: "9137871a-fadb-43ef-acb4-c5917ed17a72") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650954 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36f61295-2489-44e4-b59d-51d77bd5f40b-multus-conf-dir\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650961 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-node-log\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.650959 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64048b69-f5b1-409d-a9ec-06ffab09b71f-ovnkube-script-lib\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651017 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-etc-openvswitch\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651035 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-run\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651173 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64048b69-f5b1-409d-a9ec-06ffab09b71f-ovn-node-metrics-cert\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651259 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-log-socket\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651259 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/399ce845-2671-457b-a670-7ee180cc1d67-registration-dir\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.652531 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651301 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-slash\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.653071 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651314 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-cni-netd\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.653071 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651351 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a875e964-90d0-43e9-8da1-c427093ead9e-agent-certs\") pod \"konnectivity-agent-7f7k4\" (UID: \"a875e964-90d0-43e9-8da1-c427093ead9e\") " pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:55:38.653071 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651373 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/232c0b60-887d-41bd-8563-67bd303f57e4-host\") pod \"node-ca-5tgvd\" (UID: \"232c0b60-887d-41bd-8563-67bd303f57e4\") " pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.653071 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651379 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-sysctl-conf\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.653071 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651396 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a875e964-90d0-43e9-8da1-c427093ead9e-konnectivity-ca\") pod \"konnectivity-agent-7f7k4\" (UID: \"a875e964-90d0-43e9-8da1-c427093ead9e\") " pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:55:38.653071 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651429 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-host\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.653071 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651431 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64048b69-f5b1-409d-a9ec-06ffab09b71f-host-run-netns\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.653071 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651607 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-etc-tuned\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.653071 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.651642 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/412a8a2b-3312-4a3f-be35-dbba3ef982eb-cni-binary-copy\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.653337 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.653158 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-tmp\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.662658 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.662617 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cngl\" (UniqueName: \"kubernetes.io/projected/399ce845-2671-457b-a670-7ee180cc1d67-kube-api-access-7cngl\") pod \"aws-ebs-csi-driver-node-mnldm\" (UID: \"399ce845-2671-457b-a670-7ee180cc1d67\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.662658 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.662656 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:55:38.662846 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.662678 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:55:38.662846 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.662676 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lncmg\" (UniqueName: \"kubernetes.io/projected/36f61295-2489-44e4-b59d-51d77bd5f40b-kube-api-access-lncmg\") pod \"multus-m2hqg\" (UID: \"36f61295-2489-44e4-b59d-51d77bd5f40b\") " pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.662846 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.662693 2575 projected.go:196] Error preparing data for projected volume kube-api-access-xw8cb for pod openshift-network-diagnostics/network-check-target-slr49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:38.662846 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:38.662771 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb podName:c4deb85b-d535-425d-944d-acba732d9f2a nodeName:}" failed. No retries permitted until 2026-07-09 13:55:39.162756168 +0000 UTC m=+3.016707629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xw8cb" (UniqueName: "kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb") pod "network-check-target-slr49" (UID: "c4deb85b-d535-425d-944d-acba732d9f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:38.663245 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.663033 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s69jm\" (UniqueName: \"kubernetes.io/projected/412a8a2b-3312-4a3f-be35-dbba3ef982eb-kube-api-access-s69jm\") pod \"multus-additional-cni-plugins-zs2sz\" (UID: \"412a8a2b-3312-4a3f-be35-dbba3ef982eb\") " pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.663694 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.663671 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgxgp\" (UniqueName: \"kubernetes.io/projected/45e4c2e5-2a9c-4508-a9d1-e2c274713331-kube-api-access-tgxgp\") pod \"node-resolver-xd872\" (UID: \"45e4c2e5-2a9c-4508-a9d1-e2c274713331\") " pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.664777 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.664731 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpskm\" (UniqueName: \"kubernetes.io/projected/6584f2cb-ad76-4599-9a0c-1bc1c837e73f-kube-api-access-tpskm\") pod \"tuned-6nkn4\" (UID: \"6584f2cb-ad76-4599-9a0c-1bc1c837e73f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.664958 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.664917 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzc2q\" (UniqueName: \"kubernetes.io/projected/232c0b60-887d-41bd-8563-67bd303f57e4-kube-api-access-zzc2q\") pod \"node-ca-5tgvd\" (UID: \"232c0b60-887d-41bd-8563-67bd303f57e4\") " pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.665076 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.665057 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmf2\" (UniqueName: \"kubernetes.io/projected/b65e29e7-0841-46af-9714-e71a7444329d-kube-api-access-cbmf2\") pod \"iptables-alerter-wzgmj\" (UID: \"b65e29e7-0841-46af-9714-e71a7444329d\") " pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.665388 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.665369 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr2fh\" (UniqueName: \"kubernetes.io/projected/4e1ac675-9682-4468-993d-3d5183940a54-kube-api-access-xr2fh\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:38.665672 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.665656 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shnfd\" (UniqueName: \"kubernetes.io/projected/64048b69-f5b1-409d-a9ec-06ffab09b71f-kube-api-access-shnfd\") pod \"ovnkube-node-h84mk\" (UID: \"64048b69-f5b1-409d-a9ec-06ffab09b71f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.833039 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.832934 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" Jul 09 13:55:38.839483 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.839458 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5tgvd" Jul 09 13:55:38.844032 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.843994 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod399ce845_2671_457b_a670_7ee180cc1d67.slice/crio-84e81b5f97effc3e9f4c614fe251bf4ab39467cc6787eb99338fc2c633d9fe82 WatchSource:0}: Error finding container 84e81b5f97effc3e9f4c614fe251bf4ab39467cc6787eb99338fc2c633d9fe82: Status 404 returned error can't find the container with id 84e81b5f97effc3e9f4c614fe251bf4ab39467cc6787eb99338fc2c633d9fe82 Jul 09 13:55:38.848136 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.848111 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" Jul 09 13:55:38.852305 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.852271 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232c0b60_887d_41bd_8563_67bd303f57e4.slice/crio-cbfb0f3075672b742a7cb8e72198e9ce9a8c4bb803baa5eec512157c9604a60f WatchSource:0}: Error finding container cbfb0f3075672b742a7cb8e72198e9ce9a8c4bb803baa5eec512157c9604a60f: Status 404 returned error can't find the container with id cbfb0f3075672b742a7cb8e72198e9ce9a8c4bb803baa5eec512157c9604a60f Jul 09 13:55:38.852447 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.852371 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m2hqg" Jul 09 13:55:38.858584 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.858552 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:55:38.859232 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.859199 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod412a8a2b_3312_4a3f_be35_dbba3ef982eb.slice/crio-d5ec48396b86339c494fe4f4c4ec1860cc8632c758e02c62c9ef108fe78247d9 WatchSource:0}: Error finding container d5ec48396b86339c494fe4f4c4ec1860cc8632c758e02c62c9ef108fe78247d9: Status 404 returned error can't find the container with id d5ec48396b86339c494fe4f4c4ec1860cc8632c758e02c62c9ef108fe78247d9 Jul 09 13:55:38.862077 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.862053 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f61295_2489_44e4_b59d_51d77bd5f40b.slice/crio-955abf0a7d86c62411dfe74fbe57d7b94f618a53eb9f37e54f75283628328a1f WatchSource:0}: Error finding container 955abf0a7d86c62411dfe74fbe57d7b94f618a53eb9f37e54f75283628328a1f: Status 404 returned error can't find the container with id 955abf0a7d86c62411dfe74fbe57d7b94f618a53eb9f37e54f75283628328a1f Jul 09 13:55:38.864406 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.864379 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xd872" Jul 09 13:55:38.870405 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.870375 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda875e964_90d0_43e9_8da1_c427093ead9e.slice/crio-9e0f7142230951511b94ad77e44092c0037940b10a886e681a59f808f997015a WatchSource:0}: Error finding container 9e0f7142230951511b94ad77e44092c0037940b10a886e681a59f808f997015a: Status 404 returned error can't find the container with id 9e0f7142230951511b94ad77e44092c0037940b10a886e681a59f808f997015a Jul 09 13:55:38.872766 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.872556 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wzgmj" Jul 09 13:55:38.875336 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.875296 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45e4c2e5_2a9c_4508_a9d1_e2c274713331.slice/crio-d4e3e36a325245467fd3e78da337a751ce77a8b413ff694c9a470446b8fd2bac WatchSource:0}: Error finding container d4e3e36a325245467fd3e78da337a751ce77a8b413ff694c9a470446b8fd2bac: Status 404 returned error can't find the container with id d4e3e36a325245467fd3e78da337a751ce77a8b413ff694c9a470446b8fd2bac Jul 09 13:55:38.878020 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.877994 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:55:38.882643 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.882614 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb65e29e7_0841_46af_9714_e71a7444329d.slice/crio-303b4947e2f215ee751f7dd748677bb0bd3873bed91461254013b6ec5174ce72 WatchSource:0}: Error finding container 303b4947e2f215ee751f7dd748677bb0bd3873bed91461254013b6ec5174ce72: Status 404 returned error can't find the container with id 303b4947e2f215ee751f7dd748677bb0bd3873bed91461254013b6ec5174ce72 Jul 09 13:55:38.882742 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:38.882687 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" Jul 09 13:55:38.901371 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.901194 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64048b69_f5b1_409d_a9ec_06ffab09b71f.slice/crio-876aa6dfcc612ccdc5576cc8106a613fa9e605a660e056aaf49f511aef2f0128 WatchSource:0}: Error finding container 876aa6dfcc612ccdc5576cc8106a613fa9e605a660e056aaf49f511aef2f0128: Status 404 returned error can't find the container with id 876aa6dfcc612ccdc5576cc8106a613fa9e605a660e056aaf49f511aef2f0128 Jul 09 13:55:38.903556 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:55:38.903527 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6584f2cb_ad76_4599_9a0c_1bc1c837e73f.slice/crio-80f5ea54eb5c1e0031b84b1c8c9528b91e00604689ec0fc912726e86331cc87a WatchSource:0}: Error finding container 80f5ea54eb5c1e0031b84b1c8c9528b91e00604689ec0fc912726e86331cc87a: Status 404 returned error can't find the container with id 80f5ea54eb5c1e0031b84b1c8c9528b91e00604689ec0fc912726e86331cc87a Jul 09 13:55:39.011042 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.011002 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 13:55:39.153128 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.153026 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:39.153128 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.153086 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:39.153379 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:39.153302 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:39.153436 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:39.153382 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret podName:9137871a-fadb-43ef-acb4-c5917ed17a72 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:40.153358133 +0000 UTC m=+4.007309592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret") pod "global-pull-secret-syncer-k4zsb" (UID: "9137871a-fadb-43ef-acb4-c5917ed17a72") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:39.153901 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:39.153864 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:39.154046 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:39.153954 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:40.153918257 +0000 UTC m=+4.007869719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:39.254578 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.254519 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:39.254815 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:39.254724 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:55:39.254815 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:39.254746 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:55:39.254815 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:39.254761 2575 projected.go:196] Error preparing data for projected volume kube-api-access-xw8cb for pod openshift-network-diagnostics/network-check-target-slr49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:39.255033 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:39.254825 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb podName:c4deb85b-d535-425d-944d-acba732d9f2a nodeName:}" failed. No retries permitted until 2026-07-09 13:55:40.254805482 +0000 UTC m=+4.108756953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw8cb" (UniqueName: "kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb") pod "network-check-target-slr49" (UID: "c4deb85b-d535-425d-944d-acba732d9f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:39.593513 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.593388 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:50:38 +0000 UTC" deadline="2028-02-26 04:03:42.83045535 +0000 UTC" Jul 09 13:55:39.593513 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.593438 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14318h8m3.237024284s" Jul 09 13:55:39.632231 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.632193 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:39.632440 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:39.632350 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:39.650686 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.650642 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wzgmj" event={"ID":"b65e29e7-0841-46af-9714-e71a7444329d","Type":"ContainerStarted","Data":"303b4947e2f215ee751f7dd748677bb0bd3873bed91461254013b6ec5174ce72"} Jul 09 13:55:39.655620 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.655573 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7f7k4" event={"ID":"a875e964-90d0-43e9-8da1-c427093ead9e","Type":"ContainerStarted","Data":"9e0f7142230951511b94ad77e44092c0037940b10a886e681a59f808f997015a"} Jul 09 13:55:39.657475 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.657398 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" event={"ID":"412a8a2b-3312-4a3f-be35-dbba3ef982eb","Type":"ContainerStarted","Data":"d5ec48396b86339c494fe4f4c4ec1860cc8632c758e02c62c9ef108fe78247d9"} Jul 09 13:55:39.660066 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.660038 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5tgvd" event={"ID":"232c0b60-887d-41bd-8563-67bd303f57e4","Type":"ContainerStarted","Data":"cbfb0f3075672b742a7cb8e72198e9ce9a8c4bb803baa5eec512157c9604a60f"} Jul 09 13:55:39.662319 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.662292 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" event={"ID":"399ce845-2671-457b-a670-7ee180cc1d67","Type":"ContainerStarted","Data":"84e81b5f97effc3e9f4c614fe251bf4ab39467cc6787eb99338fc2c633d9fe82"} Jul 09 13:55:39.664181 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.664115 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" event={"ID":"6584f2cb-ad76-4599-9a0c-1bc1c837e73f","Type":"ContainerStarted","Data":"80f5ea54eb5c1e0031b84b1c8c9528b91e00604689ec0fc912726e86331cc87a"} Jul 09 13:55:39.665769 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.665629 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" event={"ID":"64048b69-f5b1-409d-a9ec-06ffab09b71f","Type":"ContainerStarted","Data":"876aa6dfcc612ccdc5576cc8106a613fa9e605a660e056aaf49f511aef2f0128"} Jul 09 13:55:39.667956 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.667929 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xd872" event={"ID":"45e4c2e5-2a9c-4508-a9d1-e2c274713331","Type":"ContainerStarted","Data":"d4e3e36a325245467fd3e78da337a751ce77a8b413ff694c9a470446b8fd2bac"} Jul 09 13:55:39.673070 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:39.673040 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m2hqg" event={"ID":"36f61295-2489-44e4-b59d-51d77bd5f40b","Type":"ContainerStarted","Data":"955abf0a7d86c62411dfe74fbe57d7b94f618a53eb9f37e54f75283628328a1f"} Jul 09 13:55:40.161304 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:40.160426 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:40.161304 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:40.160482 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:40.161304 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.160664 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:40.161304 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.160736 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret podName:9137871a-fadb-43ef-acb4-c5917ed17a72 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:42.160715599 +0000 UTC m=+6.014667071 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret") pod "global-pull-secret-syncer-k4zsb" (UID: "9137871a-fadb-43ef-acb4-c5917ed17a72") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:40.161304 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.160822 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:40.161304 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.160859 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:42.160846776 +0000 UTC m=+6.014798234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:40.261410 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:40.261368 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:40.261627 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.261607 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:55:40.261714 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.261635 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:55:40.261714 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.261649 2575 projected.go:196] Error preparing data for projected volume kube-api-access-xw8cb for pod openshift-network-diagnostics/network-check-target-slr49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:40.261819 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.261716 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb podName:c4deb85b-d535-425d-944d-acba732d9f2a nodeName:}" failed. No retries permitted until 2026-07-09 13:55:42.261696019 +0000 UTC m=+6.115647478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw8cb" (UniqueName: "kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb") pod "network-check-target-slr49" (UID: "c4deb85b-d535-425d-944d-acba732d9f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:40.594737 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:40.594604 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:50:38 +0000 UTC" deadline="2028-04-19 03:56:50.802684102 +0000 UTC" Jul 09 13:55:40.594737 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:40.594679 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15590h1m10.208010152s" Jul 09 13:55:40.635288 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:40.635094 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:40.635288 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.635246 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:40.635931 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:40.635831 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:40.636029 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:40.635977 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:41.632080 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:41.632040 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:41.632723 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:41.632191 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:42.177790 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:42.177748 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:42.178003 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:42.177807 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:42.178003 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.177983 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:42.178154 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.178048 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret podName:9137871a-fadb-43ef-acb4-c5917ed17a72 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:46.178029168 +0000 UTC m=+10.031980628 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret") pod "global-pull-secret-syncer-k4zsb" (UID: "9137871a-fadb-43ef-acb4-c5917ed17a72") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:42.178540 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.178520 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:42.178648 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.178578 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:46.17856148 +0000 UTC m=+10.032512948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:42.279166 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:42.279123 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:42.279885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.279359 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:55:42.279885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.279393 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:55:42.279885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.279409 2575 projected.go:196] Error preparing data for projected volume kube-api-access-xw8cb for pod openshift-network-diagnostics/network-check-target-slr49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:42.279885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.279474 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb podName:c4deb85b-d535-425d-944d-acba732d9f2a nodeName:}" failed. No retries permitted until 2026-07-09 13:55:46.279453372 +0000 UTC m=+10.133404829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw8cb" (UniqueName: "kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb") pod "network-check-target-slr49" (UID: "c4deb85b-d535-425d-944d-acba732d9f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:42.634419 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:42.633819 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:42.634419 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.633979 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:42.634419 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:42.634027 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:42.634419 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:42.634143 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:43.631709 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:43.631670 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:43.631940 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:43.631808 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:44.632566 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:44.632052 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:44.632566 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:44.632199 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:44.632566 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:44.632249 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:44.632566 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:44.632380 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:45.632221 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:45.632187 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:45.632418 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:45.632320 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:46.211107 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:46.211068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:46.211635 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:46.211121 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:46.211635 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.211276 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:46.211635 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.211282 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:46.211635 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.211333 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret podName:9137871a-fadb-43ef-acb4-c5917ed17a72 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:54.211316613 +0000 UTC m=+18.065268068 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret") pod "global-pull-secret-syncer-k4zsb" (UID: "9137871a-fadb-43ef-acb4-c5917ed17a72") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:46.211635 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.211354 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 13:55:54.211342535 +0000 UTC m=+18.065293995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:46.311983 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:46.311944 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:46.312187 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.312095 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:55:46.312187 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.312121 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:55:46.312187 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.312135 2575 projected.go:196] Error preparing data for projected volume kube-api-access-xw8cb for pod openshift-network-diagnostics/network-check-target-slr49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:46.312341 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.312197 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb podName:c4deb85b-d535-425d-944d-acba732d9f2a nodeName:}" failed. No retries permitted until 2026-07-09 13:55:54.312178025 +0000 UTC m=+18.166129479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw8cb" (UniqueName: "kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb") pod "network-check-target-slr49" (UID: "c4deb85b-d535-425d-944d-acba732d9f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:46.647516 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:46.647478 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:46.647690 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.647589 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:46.648012 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:46.647983 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:46.648132 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:46.648112 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:47.631556 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:47.631519 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:47.632064 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:47.631748 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:48.635519 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:48.634839 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:48.635519 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:48.634993 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:48.635519 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:48.635383 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:48.635519 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:48.635470 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:49.632039 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:49.631999 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:49.632215 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:49.632115 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:50.635096 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:50.635064 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:50.635510 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:50.635210 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:50.635510 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:50.635228 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:50.635510 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:50.635358 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:51.632320 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:51.632284 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:51.632505 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:51.632413 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:52.634803 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:52.634773 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:52.635224 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:52.634775 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:52.635224 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:52.634923 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:52.635224 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:52.634977 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:53.631333 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:53.631293 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:53.631519 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:53.631419 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:54.277998 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:54.277944 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:54.277998 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:54.277997 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:54.278586 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.278103 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:54.278586 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.278139 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:54.278586 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.278185 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:10.278167247 +0000 UTC m=+34.132118720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:55:54.278586 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.278201 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret podName:9137871a-fadb-43ef-acb4-c5917ed17a72 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:10.278195595 +0000 UTC m=+34.132147052 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret") pod "global-pull-secret-syncer-k4zsb" (UID: "9137871a-fadb-43ef-acb4-c5917ed17a72") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:55:54.379119 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:54.379080 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:54.379334 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.379238 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:55:54.379334 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.379260 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:55:54.379334 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.379272 2575 projected.go:196] Error preparing data for projected volume kube-api-access-xw8cb for pod openshift-network-diagnostics/network-check-target-slr49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:54.379464 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.379342 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb podName:c4deb85b-d535-425d-944d-acba732d9f2a nodeName:}" failed. No retries permitted until 2026-07-09 13:56:10.37932206 +0000 UTC m=+34.233273543 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw8cb" (UniqueName: "kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb") pod "network-check-target-slr49" (UID: "c4deb85b-d535-425d-944d-acba732d9f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:55:54.632684 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:54.632085 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:54.632684 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.632247 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:54.632684 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:54.632289 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:54.632684 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:54.632408 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:55.631586 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:55.631536 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:55.632018 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:55.631659 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:56.633018 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:56.632978 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:56.633483 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:56.633101 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:56.633483 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:56.633143 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:56.633483 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:56.633250 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:57.631997 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:57.631956 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:57.632204 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:57.632093 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:58.631947 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:58.631911 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:55:58.632470 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:58.631961 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:55:58.632470 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:58.632043 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:55:58.632470 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:58.632190 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:55:59.632454 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:59.632251 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:55:59.633087 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:55:59.632569 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:55:59.719876 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:59.719830 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal" event={"ID":"b3de1e4a975b796018b205e668ce5af6","Type":"ContainerStarted","Data":"b01d8119fed2521fcc9460a7af35a94b8c0f3db262aa35bbeba088f218eec9f5"} Jul 09 13:55:59.721465 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:59.721439 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" event={"ID":"6584f2cb-ad76-4599-9a0c-1bc1c837e73f","Type":"ContainerStarted","Data":"9a6b73430ffe627a7613d1aff006a5420ff6e99459ca21ce47420773aaa39221"} Jul 09 13:55:59.727169 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:59.727137 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" event={"ID":"64048b69-f5b1-409d-a9ec-06ffab09b71f","Type":"ContainerStarted","Data":"cce3f8bfe93ef03168f615de3750408ad7fc3af469f8262a67664fc42c70f069"} Jul 09 13:55:59.727312 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:59.727206 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" event={"ID":"64048b69-f5b1-409d-a9ec-06ffab09b71f","Type":"ContainerStarted","Data":"b2ebaad596d688a060cd326cd9a4d04ee140c3544f42cd97c71f5c15d8bc0562"} Jul 09 13:55:59.728891 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:59.728843 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m2hqg" event={"ID":"36f61295-2489-44e4-b59d-51d77bd5f40b","Type":"ContainerStarted","Data":"694a2734e5e5869a105278abf727b5060f77a418ff557e9ba5ad41d35ef10958"} Jul 09 13:55:59.736379 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:59.736333 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-190.ec2.internal" podStartSLOduration=21.736318845 podStartE2EDuration="21.736318845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:55:59.736050202 +0000 UTC m=+23.590001679" watchObservedRunningTime="2026-07-09 13:55:59.736318845 +0000 UTC m=+23.590270321" Jul 09 13:55:59.755236 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:55:59.755187 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-m2hqg" podStartSLOduration=2.178777652 podStartE2EDuration="22.7551693s" totalImagesPullingTime="20.576391648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:55:59.754675272 +0000 UTC m=+23.608626751" watchObservedRunningTime="2026-07-09 13:55:59.7551693 +0000 UTC m=+23.609120775" Jul 09 13:56:00.632479 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.632293 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:00.633104 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.632359 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:00.633104 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:00.632582 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:56:00.633104 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:00.632641 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:56:00.733539 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.733432 2575 generic.go:356] "Generic (PLEG): container finished" podID="1e2ef5c33e6feccabcbd54c585584e1d" containerID="7d343efbf9e2e105cf732260251d92239eaf77addc329759c3101c46408b3089" exitCode=0 Jul 09 13:56:00.733539 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.733531 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" event={"ID":"1e2ef5c33e6feccabcbd54c585584e1d","Type":"ContainerDied","Data":"7d343efbf9e2e105cf732260251d92239eaf77addc329759c3101c46408b3089"} Jul 09 13:56:00.735454 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.735425 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7f7k4" event={"ID":"a875e964-90d0-43e9-8da1-c427093ead9e","Type":"ContainerStarted","Data":"397042edacec585c596913b30e95fe4e087ae0500ca7898af5ad96c89dd75f6d"} Jul 09 13:56:00.737383 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.737356 2575 generic.go:356] "Generic (PLEG): container finished" podID="412a8a2b-3312-4a3f-be35-dbba3ef982eb" containerID="f99140a811f6ef11f621bc3b33db1134d47fcfa8c6c39b2a65c23267722a9220" exitCode=0 Jul 09 13:56:00.737508 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.737433 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" event={"ID":"412a8a2b-3312-4a3f-be35-dbba3ef982eb","Type":"ContainerDied","Data":"f99140a811f6ef11f621bc3b33db1134d47fcfa8c6c39b2a65c23267722a9220"} Jul 09 13:56:00.739433 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.739405 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5tgvd" event={"ID":"232c0b60-887d-41bd-8563-67bd303f57e4","Type":"ContainerStarted","Data":"3ef1a27acfbdc6e43e857520a87117ffd255f750bb220a280b6f88afa4ceb62e"} Jul 09 13:56:00.741345 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.741318 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" event={"ID":"399ce845-2671-457b-a670-7ee180cc1d67","Type":"ContainerStarted","Data":"e53e3855c55e54b2c6b56569fad51e077b5075d3d8fbacf5f9ccb061e7ffe68a"} Jul 09 13:56:00.745210 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.745111 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" event={"ID":"64048b69-f5b1-409d-a9ec-06ffab09b71f","Type":"ContainerStarted","Data":"96c0cd4c3739b9c382b08e6787989c3d82ac51ded728985e07b44210b33a57d2"} Jul 09 13:56:00.745210 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.745190 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" event={"ID":"64048b69-f5b1-409d-a9ec-06ffab09b71f","Type":"ContainerStarted","Data":"c9ca686facba1464e02baf800526033663c17b5b4183b5a1f5c904699b96c9cd"} Jul 09 13:56:00.745298 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.745228 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" event={"ID":"64048b69-f5b1-409d-a9ec-06ffab09b71f","Type":"ContainerStarted","Data":"0392632a6a156a13e492a03e725527e46e94740c7599e1891d9eef4f3b9770ec"} Jul 09 13:56:00.745298 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.745273 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" event={"ID":"64048b69-f5b1-409d-a9ec-06ffab09b71f","Type":"ContainerStarted","Data":"4c13f4e05c0e7c1622dc682e80d287da69d5b3e96754915e7f9af71fd2227e92"} Jul 09 13:56:00.746779 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.746751 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xd872" event={"ID":"45e4c2e5-2a9c-4508-a9d1-e2c274713331","Type":"ContainerStarted","Data":"22524f265f64ea75950dc8ac1e4ed3b035e3c41277d2ad72ba118fd42de926cd"} Jul 09 13:56:00.750660 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.750619 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6nkn4" podStartSLOduration=3.263324681 podStartE2EDuration="23.750606452s" totalImagesPullingTime="20.487281771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:55:59.772783331 +0000 UTC m=+23.626734801" watchObservedRunningTime="2026-07-09 13:56:00.750606452 +0000 UTC m=+24.604557946" Jul 09 13:56:00.766671 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.766631 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7f7k4" podStartSLOduration=3.33451249 podStartE2EDuration="23.76661126s" totalImagesPullingTime="20.43209877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:00.766371731 +0000 UTC m=+24.620323220" watchObservedRunningTime="2026-07-09 13:56:00.76661126 +0000 UTC m=+24.620562751" Jul 09 13:56:00.783047 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.782988 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5tgvd" podStartSLOduration=3.310497138 podStartE2EDuration="23.782966302s" totalImagesPullingTime="20.472469164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:00.782470334 +0000 UTC m=+24.636421819" watchObservedRunningTime="2026-07-09 13:56:00.782966302 +0000 UTC m=+24.636917777" Jul 09 13:56:00.801318 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:00.801252 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xd872" podStartSLOduration=7.437154468 podStartE2EDuration="23.801229768s" totalImagesPullingTime="16.3640753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:00.801193105 +0000 UTC m=+24.655144581" watchObservedRunningTime="2026-07-09 13:56:00.801229768 +0000 UTC m=+24.655181248" Jul 09 13:56:01.510397 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.510369 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 13:56:01.616933 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.616774 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T13:56:01.510388934Z","UUID":"ba25037a-dce9-4cc8-9112-80ee9c40278f","Handler":null,"Name":"","Endpoint":""} Jul 09 13:56:01.618247 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.618226 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 13:56:01.618379 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.618268 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 13:56:01.631703 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.631669 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:01.631892 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:01.631768 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:56:01.751405 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.751320 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" event={"ID":"1e2ef5c33e6feccabcbd54c585584e1d","Type":"ContainerStarted","Data":"7ba67117cb5c39bc7b5dc9bf0ba9e0e9667d4474f4410e3123df6f7aecae1bc1"} Jul 09 13:56:01.752820 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.752797 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wzgmj" event={"ID":"b65e29e7-0841-46af-9714-e71a7444329d","Type":"ContainerStarted","Data":"f8feb5628db6c0254608302d00bbe786178bea60fcf3cf5efad116144da3273b"} Jul 09 13:56:01.754555 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.754533 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" event={"ID":"399ce845-2671-457b-a670-7ee180cc1d67","Type":"ContainerStarted","Data":"c57ab6367a318064354f5df0e3d8b38d27608be8915bbf59d6e9973f505fdfea"} Jul 09 13:56:01.768167 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.768111 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-190.ec2.internal" podStartSLOduration=23.768095197 podStartE2EDuration="23.768095197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:01.767734259 +0000 UTC m=+25.621685735" watchObservedRunningTime="2026-07-09 13:56:01.768095197 +0000 UTC m=+25.622046672" Jul 09 13:56:01.784803 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:01.784750 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wzgmj" podStartSLOduration=4.342216973 podStartE2EDuration="24.784732554s" totalImagesPullingTime="20.442515581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:01.78429825 +0000 UTC m=+25.638249727" watchObservedRunningTime="2026-07-09 13:56:01.784732554 +0000 UTC m=+25.638684081" Jul 09 13:56:02.631655 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:02.631605 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:02.631655 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:02.631651 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:02.631969 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:02.631759 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:56:02.631969 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:02.631937 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:56:02.762628 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:02.762592 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" event={"ID":"64048b69-f5b1-409d-a9ec-06ffab09b71f","Type":"ContainerStarted","Data":"1afb60ccca2cd8daae677ae6ab7453d5fa85dd5530b08967806cb3a300f83813"} Jul 09 13:56:03.632245 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:03.632210 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:03.632443 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:03.632378 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:56:03.768079 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:03.768036 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" event={"ID":"399ce845-2671-457b-a670-7ee180cc1d67","Type":"ContainerStarted","Data":"97860f88b8ea262b74c147d17771da5910d6577765eb0f333278b8cd8ea5d061"} Jul 09 13:56:03.788408 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:03.788354 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mnldm" podStartSLOduration=3.817823984 podStartE2EDuration="26.788337362s" totalImagesPullingTime="22.970513378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:03.787865826 +0000 UTC m=+27.641817303" watchObservedRunningTime="2026-07-09 13:56:03.788337362 +0000 UTC m=+27.642288842" Jul 09 13:56:04.181197 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:04.181151 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:56:04.181990 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:04.181970 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:56:04.631675 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:04.631440 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:04.631897 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:04.631487 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:04.631897 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:04.631785 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:56:04.631897 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:04.631863 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:56:04.770728 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:04.770496 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:56:04.771182 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:04.770885 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7f7k4" Jul 09 13:56:05.631750 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:05.631715 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:05.631968 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:05.631841 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:56:06.632656 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:06.632482 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:06.633516 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:06.632529 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:06.633516 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:06.632738 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:56:06.633516 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:06.632865 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:56:06.777541 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:06.777505 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" event={"ID":"64048b69-f5b1-409d-a9ec-06ffab09b71f","Type":"ContainerStarted","Data":"716b72af9c409f6b3f9c8a849f511ca7c48e5735049d26adc946ae849fd12f18"} Jul 09 13:56:06.777763 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:06.777713 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:56:06.779640 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:06.779616 2575 generic.go:356] "Generic (PLEG): container finished" podID="412a8a2b-3312-4a3f-be35-dbba3ef982eb" containerID="b04ca292819fc85d81ed3f205fb9d18c3615e611d09ebc0ff4e67ac1733b18c2" exitCode=0 Jul 09 13:56:06.779772 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:06.779700 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" event={"ID":"412a8a2b-3312-4a3f-be35-dbba3ef982eb","Type":"ContainerDied","Data":"b04ca292819fc85d81ed3f205fb9d18c3615e611d09ebc0ff4e67ac1733b18c2"} Jul 09 13:56:06.795504 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:06.795476 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:56:06.833626 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:06.833573 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" podStartSLOduration=9.325084916 podStartE2EDuration="29.833553846s" totalImagesPullingTime="20.50846893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:06.810463535 +0000 UTC m=+30.664415012" watchObservedRunningTime="2026-07-09 13:56:06.833553846 +0000 UTC m=+30.687505383" Jul 09 13:56:07.325251 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:07.325211 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:56:07.632371 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:07.632289 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:07.632510 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:07.632403 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:56:07.782692 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:07.782661 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:56:07.798962 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:07.798934 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:56:08.352267 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:08.352226 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k4zsb"] Jul 09 13:56:08.352486 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:08.352355 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:08.352486 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:08.352451 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:56:08.355492 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:08.355465 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t2rdv"] Jul 09 13:56:08.355610 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:08.355572 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:08.355672 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:08.355654 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:56:08.360107 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:08.360066 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-slr49"] Jul 09 13:56:08.360202 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:08.360166 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:08.360276 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:08.360258 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:56:08.786532 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:08.786496 2575 generic.go:356] "Generic (PLEG): container finished" podID="412a8a2b-3312-4a3f-be35-dbba3ef982eb" containerID="9ea6dbc753f64292e1f13501a9afd248a0ae0e241f50987a89c4b2ba2bdd1e30" exitCode=0 Jul 09 13:56:08.786946 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:08.786627 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" event={"ID":"412a8a2b-3312-4a3f-be35-dbba3ef982eb","Type":"ContainerDied","Data":"9ea6dbc753f64292e1f13501a9afd248a0ae0e241f50987a89c4b2ba2bdd1e30"} Jul 09 13:56:09.791196 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:09.791156 2575 generic.go:356] "Generic (PLEG): container finished" podID="412a8a2b-3312-4a3f-be35-dbba3ef982eb" containerID="5ae1a71b69ad02b014ac33c353586d675ac1b682e2f72c53532d866db153cfaf" exitCode=0 Jul 09 13:56:09.791671 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:09.791259 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" event={"ID":"412a8a2b-3312-4a3f-be35-dbba3ef982eb","Type":"ContainerDied","Data":"5ae1a71b69ad02b014ac33c353586d675ac1b682e2f72c53532d866db153cfaf"} Jul 09 13:56:10.297249 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:10.297078 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:10.297249 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:10.297258 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:10.297490 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.297232 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:56:10.297490 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.297369 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 13:56:10.297490 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.297372 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:42.297355089 +0000 UTC m=+66.151306553 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 13:56:10.297490 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.297418 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret podName:9137871a-fadb-43ef-acb4-c5917ed17a72 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:42.297405371 +0000 UTC m=+66.151356824 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret") pod "global-pull-secret-syncer-k4zsb" (UID: "9137871a-fadb-43ef-acb4-c5917ed17a72") : object "kube-system"/"original-pull-secret" not registered Jul 09 13:56:10.397761 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:10.397661 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:10.397940 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.397807 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 13:56:10.397940 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.397834 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 13:56:10.397940 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.397866 2575 projected.go:196] Error preparing data for projected volume kube-api-access-xw8cb for pod openshift-network-diagnostics/network-check-target-slr49: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:56:10.397940 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.397930 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb podName:c4deb85b-d535-425d-944d-acba732d9f2a nodeName:}" failed. No retries permitted until 2026-07-09 13:56:42.397912616 +0000 UTC m=+66.251864087 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xw8cb" (UniqueName: "kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb") pod "network-check-target-slr49" (UID: "c4deb85b-d535-425d-944d-acba732d9f2a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 13:56:10.635673 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:10.635646 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:10.635885 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:10.635687 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:10.635885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.635744 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:56:10.635885 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:10.635687 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:10.635885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.635802 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:56:10.636112 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:10.635918 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:56:12.635162 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:12.635129 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:12.635602 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:12.635138 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:12.635602 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:12.635256 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k4zsb" podUID="9137871a-fadb-43ef-acb4-c5917ed17a72" Jul 09 13:56:12.635602 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:12.635142 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:12.635602 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:12.635327 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-slr49" podUID="c4deb85b-d535-425d-944d-acba732d9f2a" Jul 09 13:56:12.635602 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:12.635409 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:56:13.498332 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.498303 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-190.ec2.internal" event="NodeReady" Jul 09 13:56:13.498507 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.498451 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 13:56:13.535119 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.535085 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-958d69fff-j2glx"] Jul 09 13:56:13.537759 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.537724 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.538600 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.538569 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-cngfc"] Jul 09 13:56:13.540304 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.540282 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:13.541594 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.541574 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 13:56:13.541713 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.541596 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 13:56:13.541713 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.541597 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-blxcx\"" Jul 09 13:56:13.542175 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.542018 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 13:56:13.544206 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.544180 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6tr79\"" Jul 09 13:56:13.544315 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.544216 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 13:56:13.544315 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.544186 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 13:56:13.548069 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.548048 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 13:56:13.549409 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.549389 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gnkr9"] Jul 09 13:56:13.551442 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.551421 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.553234 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.553211 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-cngfc"] Jul 09 13:56:13.554485 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.554465 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 13:56:13.554584 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.554539 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 13:56:13.554819 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.554803 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dkvmm\"" Jul 09 13:56:13.562628 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.560953 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-958d69fff-j2glx"] Jul 09 13:56:13.562628 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.561789 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5xqw6"] Jul 09 13:56:13.565414 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.565393 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gnkr9"] Jul 09 13:56:13.565529 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.565514 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5xqw6" Jul 09 13:56:13.569041 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.569016 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 13:56:13.569431 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.569027 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 13:56:13.569569 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.569319 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 13:56:13.569656 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.569321 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-r54c2\"" Jul 09 13:56:13.574821 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.574799 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5xqw6"] Jul 09 13:56:13.620897 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.620859 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-image-registry-private-configuration\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.621079 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.620917 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-installation-pull-secrets\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.621079 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.620984 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-ca-trust-extracted\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.621079 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621018 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.621079 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621042 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:13.621079 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621058 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/92ccdc79-fdc6-48fe-a976-23b6362217c1-config-volume\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.621323 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621170 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ccdc79-fdc6-48fe-a976-23b6362217c1-tmp-dir\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.621323 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621209 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-certificates\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.621323 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621249 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq2t7\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-kube-api-access-dq2t7\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.621323 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621287 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n44m\" (UniqueName: \"kubernetes.io/projected/92ccdc79-fdc6-48fe-a976-23b6362217c1-kube-api-access-9n44m\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.621515 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621343 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-bound-sa-token\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.621515 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621387 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-trusted-ca\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.621515 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621416 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:13.621515 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.621460 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.722795 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.722755 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-trusted-ca\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.722801 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.722844 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.722891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fl2p\" (UniqueName: \"kubernetes.io/projected/48bc75ef-a454-4f16-8861-a3ca613afd68-kube-api-access-9fl2p\") pod \"ingress-canary-5xqw6\" (UID: \"48bc75ef-a454-4f16-8861-a3ca613afd68\") " pod="openshift-ingress-canary/ingress-canary-5xqw6" Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.722930 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-image-registry-private-configuration\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.722955 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-installation-pull-secrets\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.722987 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-ca-trust-extracted\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723018 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:13.723022 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723046 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48bc75ef-a454-4f16-8861-a3ca613afd68-cert\") pod \"ingress-canary-5xqw6\" (UID: \"48bc75ef-a454-4f16-8861-a3ca613afd68\") " pod="openshift-ingress-canary/ingress-canary-5xqw6" Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:13.723108 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert podName:e836fc41-ecd7-4b4d-83c1-55f05ec601bd nodeName:}" failed. No retries permitted until 2026-07-09 13:56:14.223082552 +0000 UTC m=+38.077034006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cngfc" (UID: "e836fc41-ecd7-4b4d-83c1-55f05ec601bd") : secret "networking-console-plugin-cert" not found Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:13.723027 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:13.723128 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-958d69fff-j2glx: secret "image-registry-tls" not found Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:13.723145 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 13:56:13.723385 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:13.723161 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls podName:0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:14.223149611 +0000 UTC m=+38.077101066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls") pod "image-registry-958d69fff-j2glx" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2") : secret "image-registry-tls" not found Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723491 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723518 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-ca-trust-extracted\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723529 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/92ccdc79-fdc6-48fe-a976-23b6362217c1-config-volume\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723598 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ccdc79-fdc6-48fe-a976-23b6362217c1-tmp-dir\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:13.723629 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls podName:92ccdc79-fdc6-48fe-a976-23b6362217c1 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:14.223611741 +0000 UTC m=+38.077563198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls") pod "dns-default-gnkr9" (UID: "92ccdc79-fdc6-48fe-a976-23b6362217c1") : secret "dns-default-metrics-tls" not found Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723650 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-certificates\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723768 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq2t7\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-kube-api-access-dq2t7\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723817 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n44m\" (UniqueName: \"kubernetes.io/projected/92ccdc79-fdc6-48fe-a976-23b6362217c1-kube-api-access-9n44m\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723881 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-bound-sa-token\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723906 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-trusted-ca\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.724102 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.723963 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92ccdc79-fdc6-48fe-a976-23b6362217c1-tmp-dir\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.724575 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.724146 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-certificates\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.724575 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.724267 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:13.724575 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.724552 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/92ccdc79-fdc6-48fe-a976-23b6362217c1-config-volume\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.728612 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.728585 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-image-registry-private-configuration\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.728612 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.728597 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-installation-pull-secrets\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.736178 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.736137 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-bound-sa-token\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.736347 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.736240 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq2t7\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-kube-api-access-dq2t7\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:13.738328 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.738304 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n44m\" (UniqueName: \"kubernetes.io/projected/92ccdc79-fdc6-48fe-a976-23b6362217c1-kube-api-access-9n44m\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:13.825090 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.825057 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fl2p\" (UniqueName: \"kubernetes.io/projected/48bc75ef-a454-4f16-8861-a3ca613afd68-kube-api-access-9fl2p\") pod \"ingress-canary-5xqw6\" (UID: \"48bc75ef-a454-4f16-8861-a3ca613afd68\") " pod="openshift-ingress-canary/ingress-canary-5xqw6" Jul 09 13:56:13.825256 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.825113 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48bc75ef-a454-4f16-8861-a3ca613afd68-cert\") pod \"ingress-canary-5xqw6\" (UID: \"48bc75ef-a454-4f16-8861-a3ca613afd68\") " pod="openshift-ingress-canary/ingress-canary-5xqw6" Jul 09 13:56:13.828248 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.828222 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/48bc75ef-a454-4f16-8861-a3ca613afd68-cert\") pod \"ingress-canary-5xqw6\" (UID: \"48bc75ef-a454-4f16-8861-a3ca613afd68\") " pod="openshift-ingress-canary/ingress-canary-5xqw6" Jul 09 13:56:13.834905 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.834843 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fl2p\" (UniqueName: \"kubernetes.io/projected/48bc75ef-a454-4f16-8861-a3ca613afd68-kube-api-access-9fl2p\") pod \"ingress-canary-5xqw6\" (UID: \"48bc75ef-a454-4f16-8861-a3ca613afd68\") " pod="openshift-ingress-canary/ingress-canary-5xqw6" Jul 09 13:56:13.875048 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:13.875012 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5xqw6" Jul 09 13:56:14.028392 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.028204 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5xqw6"] Jul 09 13:56:14.033003 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:56:14.032976 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48bc75ef_a454_4f16_8861_a3ca613afd68.slice/crio-6239b32cf8abfbc361b356039c4a5f6999af0a2606a0c1d245244e39c29c8022 WatchSource:0}: Error finding container 6239b32cf8abfbc361b356039c4a5f6999af0a2606a0c1d245244e39c29c8022: Status 404 returned error can't find the container with id 6239b32cf8abfbc361b356039c4a5f6999af0a2606a0c1d245244e39c29c8022 Jul 09 13:56:14.229754 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.229710 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:14.229953 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.229784 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:14.229953 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.229820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:14.229953 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:14.229899 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 13:56:14.229953 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:14.229923 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 13:56:14.229953 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:14.229934 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-958d69fff-j2glx: secret "image-registry-tls" not found Jul 09 13:56:14.230130 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:14.229899 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 13:56:14.230130 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:14.229977 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert podName:e836fc41-ecd7-4b4d-83c1-55f05ec601bd nodeName:}" failed. No retries permitted until 2026-07-09 13:56:15.22995423 +0000 UTC m=+39.083905684 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cngfc" (UID: "e836fc41-ecd7-4b4d-83c1-55f05ec601bd") : secret "networking-console-plugin-cert" not found Jul 09 13:56:14.230130 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:14.229996 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls podName:92ccdc79-fdc6-48fe-a976-23b6362217c1 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:15.229984827 +0000 UTC m=+39.083936281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls") pod "dns-default-gnkr9" (UID: "92ccdc79-fdc6-48fe-a976-23b6362217c1") : secret "dns-default-metrics-tls" not found Jul 09 13:56:14.230130 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:14.230007 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls podName:0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:15.230001371 +0000 UTC m=+39.083952825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls") pod "image-registry-958d69fff-j2glx" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2") : secret "image-registry-tls" not found Jul 09 13:56:14.634798 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.634767 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:14.635019 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.634867 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:14.635085 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.635029 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:14.638208 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.638188 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 13:56:14.638208 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.638205 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 13:56:14.638391 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.638205 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 13:56:14.638391 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.638252 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 13:56:14.638978 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.638963 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qkthd\"" Jul 09 13:56:14.639053 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.638985 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mkkhr\"" Jul 09 13:56:14.804761 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.804722 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5xqw6" event={"ID":"48bc75ef-a454-4f16-8861-a3ca613afd68","Type":"ContainerStarted","Data":"6239b32cf8abfbc361b356039c4a5f6999af0a2606a0c1d245244e39c29c8022"} Jul 09 13:56:14.807988 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.807954 2575 generic.go:356] "Generic (PLEG): container finished" podID="412a8a2b-3312-4a3f-be35-dbba3ef982eb" containerID="da5f493dfd0641950759262ae4c278d4efbabffbe68bce0273f3a63766445034" exitCode=0 Jul 09 13:56:14.808148 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:14.808046 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" event={"ID":"412a8a2b-3312-4a3f-be35-dbba3ef982eb","Type":"ContainerDied","Data":"da5f493dfd0641950759262ae4c278d4efbabffbe68bce0273f3a63766445034"} Jul 09 13:56:15.238653 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:15.238569 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:15.238653 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:15.238630 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:15.238885 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:15.238678 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:15.238885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:15.238750 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 13:56:15.238885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:15.238803 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 13:56:15.238885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:15.238815 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 13:56:15.238885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:15.238823 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-958d69fff-j2glx: secret "image-registry-tls" not found Jul 09 13:56:15.238885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:15.238842 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert podName:e836fc41-ecd7-4b4d-83c1-55f05ec601bd nodeName:}" failed. No retries permitted until 2026-07-09 13:56:17.238819456 +0000 UTC m=+41.092770913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cngfc" (UID: "e836fc41-ecd7-4b4d-83c1-55f05ec601bd") : secret "networking-console-plugin-cert" not found Jul 09 13:56:15.238885 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:15.238889 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls podName:0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:17.238872264 +0000 UTC m=+41.092823732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls") pod "image-registry-958d69fff-j2glx" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2") : secret "image-registry-tls" not found Jul 09 13:56:15.239165 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:15.238904 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls podName:92ccdc79-fdc6-48fe-a976-23b6362217c1 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:17.238897961 +0000 UTC m=+41.092849415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls") pod "dns-default-gnkr9" (UID: "92ccdc79-fdc6-48fe-a976-23b6362217c1") : secret "dns-default-metrics-tls" not found Jul 09 13:56:15.817277 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:15.817219 2575 generic.go:356] "Generic (PLEG): container finished" podID="412a8a2b-3312-4a3f-be35-dbba3ef982eb" containerID="843935e97a741ee82d67e38c8e83108421408717787d95d99888ebfdeaf46fb9" exitCode=0 Jul 09 13:56:15.817632 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:15.817308 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" event={"ID":"412a8a2b-3312-4a3f-be35-dbba3ef982eb","Type":"ContainerDied","Data":"843935e97a741ee82d67e38c8e83108421408717787d95d99888ebfdeaf46fb9"} Jul 09 13:56:16.823715 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:16.823683 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" event={"ID":"412a8a2b-3312-4a3f-be35-dbba3ef982eb","Type":"ContainerStarted","Data":"4cd06a869585e77a731c20bae4a0c3746a5711ca616667859298d377ede7b846"} Jul 09 13:56:16.853961 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:16.853828 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zs2sz" podStartSLOduration=7.884970793 podStartE2EDuration="39.853812359s" totalImagesPullingTime="31.968841566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:16.852442998 +0000 UTC m=+40.706394513" watchObservedRunningTime="2026-07-09 13:56:16.853812359 +0000 UTC m=+40.707763834" Jul 09 13:56:17.014954 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:17.014734 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" limit=1024 Jul 09 13:56:17.155345 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:17.155242 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" limit=1024 Jul 09 13:56:17.255230 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:17.255187 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:17.255466 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:17.255290 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:17.255466 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:17.255334 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:17.255466 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:17.255350 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 13:56:17.255466 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:17.255415 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls podName:92ccdc79-fdc6-48fe-a976-23b6362217c1 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:21.255398997 +0000 UTC m=+45.109350454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls") pod "dns-default-gnkr9" (UID: "92ccdc79-fdc6-48fe-a976-23b6362217c1") : secret "dns-default-metrics-tls" not found Jul 09 13:56:17.255466 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:17.255432 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 13:56:17.255643 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:17.255491 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert podName:e836fc41-ecd7-4b4d-83c1-55f05ec601bd nodeName:}" failed. No retries permitted until 2026-07-09 13:56:21.255478631 +0000 UTC m=+45.109430089 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cngfc" (UID: "e836fc41-ecd7-4b4d-83c1-55f05ec601bd") : secret "networking-console-plugin-cert" not found Jul 09 13:56:17.255643 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:17.255438 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 13:56:17.255643 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:17.255509 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-958d69fff-j2glx: secret "image-registry-tls" not found Jul 09 13:56:17.255643 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:17.255537 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls podName:0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:21.255530289 +0000 UTC m=+45.109481743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls") pod "image-registry-958d69fff-j2glx" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2") : secret "image-registry-tls" not found Jul 09 13:56:17.828037 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:17.828001 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5xqw6" event={"ID":"48bc75ef-a454-4f16-8861-a3ca613afd68","Type":"ContainerStarted","Data":"8cd250b82bc2d61d7424599588e66d93757da8c43e5ef3da12b1f1a6e219b54b"} Jul 09 13:56:17.848861 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:17.848805 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5xqw6" podStartSLOduration=1.9652101279999998 podStartE2EDuration="4.848789256s" totalImagesPullingTime="2.883579128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:17.847749445 +0000 UTC m=+41.701700912" watchObservedRunningTime="2026-07-09 13:56:17.848789256 +0000 UTC m=+41.702740732" Jul 09 13:56:18.027321 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:18.027277 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" limit=1024 Jul 09 13:56:18.089983 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:18.089932 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" limit=1024 Jul 09 13:56:21.293216 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:21.293174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:21.293216 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:21.293229 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:21.293725 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:21.293268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:21.293725 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:21.293344 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 13:56:21.293725 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:21.293358 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 13:56:21.293725 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:21.293380 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-958d69fff-j2glx: secret "image-registry-tls" not found Jul 09 13:56:21.293725 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:21.293354 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 13:56:21.293725 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:21.293413 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert podName:e836fc41-ecd7-4b4d-83c1-55f05ec601bd nodeName:}" failed. No retries permitted until 2026-07-09 13:56:29.293395031 +0000 UTC m=+53.147346486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cngfc" (UID: "e836fc41-ecd7-4b4d-83c1-55f05ec601bd") : secret "networking-console-plugin-cert" not found Jul 09 13:56:21.293725 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:21.293435 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls podName:0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:29.293420186 +0000 UTC m=+53.147371645 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls") pod "image-registry-958d69fff-j2glx" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2") : secret "image-registry-tls" not found Jul 09 13:56:21.293725 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:21.293449 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls podName:92ccdc79-fdc6-48fe-a976-23b6362217c1 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:29.293443752 +0000 UTC m=+53.147395206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls") pod "dns-default-gnkr9" (UID: "92ccdc79-fdc6-48fe-a976-23b6362217c1") : secret "dns-default-metrics-tls" not found Jul 09 13:56:24.374681 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:24.374647 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" limit=1024 Jul 09 13:56:24.431134 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:24.431098 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" limit=1024 Jul 09 13:56:25.926410 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:25.926376 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" limit=1024 Jul 09 13:56:29.355542 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:29.355496 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:29.355542 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:29.355548 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:29.356103 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:29.355602 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:29.356103 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:29.355660 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 13:56:29.356103 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:29.355683 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-958d69fff-j2glx: secret "image-registry-tls" not found Jul 09 13:56:29.356103 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:29.355687 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 13:56:29.356103 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:29.355706 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 13:56:29.356103 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:29.355744 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert podName:e836fc41-ecd7-4b4d-83c1-55f05ec601bd nodeName:}" failed. No retries permitted until 2026-07-09 13:56:45.35572866 +0000 UTC m=+69.209680115 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cngfc" (UID: "e836fc41-ecd7-4b4d-83c1-55f05ec601bd") : secret "networking-console-plugin-cert" not found Jul 09 13:56:29.356103 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:29.355758 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls podName:0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:45.355751261 +0000 UTC m=+69.209702715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls") pod "image-registry-958d69fff-j2glx" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2") : secret "image-registry-tls" not found Jul 09 13:56:29.356103 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:29.355767 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls podName:92ccdc79-fdc6-48fe-a976-23b6362217c1 nodeName:}" failed. No retries permitted until 2026-07-09 13:56:45.355762162 +0000 UTC m=+69.209713615 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls") pod "dns-default-gnkr9" (UID: "92ccdc79-fdc6-48fe-a976-23b6362217c1") : secret "dns-default-metrics-tls" not found Jul 09 13:56:29.884074 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:29.884042 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" limit=1024 Jul 09 13:56:29.983586 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:29.983542 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" limit=1024 Jul 09 13:56:39.804548 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:39.804517 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h84mk" Jul 09 13:56:42.358665 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.358621 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:56:42.358665 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.358668 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:42.361952 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.361924 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 13:56:42.361952 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.361941 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 13:56:42.369297 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:42.369263 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 13:56:42.369424 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:42.369360 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 13:57:46.369337426 +0000 UTC m=+130.223288880 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : secret "metrics-daemon-secret" not found Jul 09 13:56:42.373329 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.373302 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9137871a-fadb-43ef-acb4-c5917ed17a72-original-pull-secret\") pod \"global-pull-secret-syncer-k4zsb\" (UID: \"9137871a-fadb-43ef-acb4-c5917ed17a72\") " pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:42.459724 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.459667 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:42.463015 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.462988 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 13:56:42.473659 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.473629 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 13:56:42.484055 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.484022 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw8cb\" (UniqueName: \"kubernetes.io/projected/c4deb85b-d535-425d-944d-acba732d9f2a-kube-api-access-xw8cb\") pod \"network-check-target-slr49\" (UID: \"c4deb85b-d535-425d-944d-acba732d9f2a\") " pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:42.544168 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.544125 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k4zsb" Jul 09 13:56:42.558423 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.558394 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mkkhr\"" Jul 09 13:56:42.565860 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.565821 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:42.681884 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.681833 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k4zsb"] Jul 09 13:56:42.685063 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:56:42.685028 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9137871a_fadb_43ef_acb4_c5917ed17a72.slice/crio-86d05c86e021326e75f441bf04fe0dd18c36cf55e94eccc4b786fabf8ceac9dc WatchSource:0}: Error finding container 86d05c86e021326e75f441bf04fe0dd18c36cf55e94eccc4b786fabf8ceac9dc: Status 404 returned error can't find the container with id 86d05c86e021326e75f441bf04fe0dd18c36cf55e94eccc4b786fabf8ceac9dc Jul 09 13:56:42.725820 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.725777 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-slr49"] Jul 09 13:56:42.728442 ip-10-0-128-190 kubenswrapper[2575]: W0709 13:56:42.728418 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4deb85b_d535_425d_944d_acba732d9f2a.slice/crio-2f5f0f2afbd2daad4deb08fa6f70a8c78424fc260fdb583a48dcfb33126609d9 WatchSource:0}: Error finding container 2f5f0f2afbd2daad4deb08fa6f70a8c78424fc260fdb583a48dcfb33126609d9: Status 404 returned error can't find the container with id 2f5f0f2afbd2daad4deb08fa6f70a8c78424fc260fdb583a48dcfb33126609d9 Jul 09 13:56:42.889410 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.889298 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-slr49" event={"ID":"c4deb85b-d535-425d-944d-acba732d9f2a","Type":"ContainerStarted","Data":"2f5f0f2afbd2daad4deb08fa6f70a8c78424fc260fdb583a48dcfb33126609d9"} Jul 09 13:56:42.890389 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:42.890365 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k4zsb" event={"ID":"9137871a-fadb-43ef-acb4-c5917ed17a72","Type":"ContainerStarted","Data":"86d05c86e021326e75f441bf04fe0dd18c36cf55e94eccc4b786fabf8ceac9dc"} Jul 09 13:56:45.386785 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:45.386745 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:56:45.387281 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:45.386833 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:56:45.387281 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:45.386886 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:56:45.387281 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:45.386930 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 13:56:45.387281 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:45.386993 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 13:56:45.387281 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:45.386992 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 13:56:45.387281 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:45.386993 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls podName:92ccdc79-fdc6-48fe-a976-23b6362217c1 nodeName:}" failed. No retries permitted until 2026-07-09 13:57:17.386974856 +0000 UTC m=+101.240926328 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls") pod "dns-default-gnkr9" (UID: "92ccdc79-fdc6-48fe-a976-23b6362217c1") : secret "dns-default-metrics-tls" not found Jul 09 13:56:45.387281 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:45.387005 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-958d69fff-j2glx: secret "image-registry-tls" not found Jul 09 13:56:45.387281 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:45.387083 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert podName:e836fc41-ecd7-4b4d-83c1-55f05ec601bd nodeName:}" failed. No retries permitted until 2026-07-09 13:57:17.387064183 +0000 UTC m=+101.241015646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cngfc" (UID: "e836fc41-ecd7-4b4d-83c1-55f05ec601bd") : secret "networking-console-plugin-cert" not found Jul 09 13:56:45.387281 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:56:45.387128 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls podName:0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2 nodeName:}" failed. No retries permitted until 2026-07-09 13:57:17.387116621 +0000 UTC m=+101.241068103 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls") pod "image-registry-958d69fff-j2glx" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2") : secret "image-registry-tls" not found Jul 09 13:56:47.906708 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:47.906662 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-slr49" event={"ID":"c4deb85b-d535-425d-944d-acba732d9f2a","Type":"ContainerStarted","Data":"b8f129b611848a434f292c781ca69c11f3cfa939b60d069230db2a398ecbfc94"} Jul 09 13:56:47.907221 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:47.906763 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:56:47.908289 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:47.908262 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k4zsb" event={"ID":"9137871a-fadb-43ef-acb4-c5917ed17a72","Type":"ContainerStarted","Data":"70f1df44f4a3c36e85cfc36d3ce82881c483a9bd788e571fe47a8fd2d47a279c"} Jul 09 13:56:47.923064 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:47.922790 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-slr49" podStartSLOduration=66.140030164 podStartE2EDuration="1m10.922771642s" totalImagesPullingTime="4.782741478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:47.92255644 +0000 UTC m=+71.776507917" watchObservedRunningTime="2026-07-09 13:56:47.922771642 +0000 UTC m=+71.776723118" Jul 09 13:56:47.939979 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:56:47.939812 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k4zsb" podStartSLOduration=65.883617965 podStartE2EDuration="1m10.939790897s" totalImagesPullingTime="5.056172932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:56:47.93956319 +0000 UTC m=+71.793514666" watchObservedRunningTime="2026-07-09 13:56:47.939790897 +0000 UTC m=+71.793742374" Jul 09 13:57:17.425845 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:57:17.425702 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:57:17.425845 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:57:17.425767 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:57:17.425845 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:57:17.425801 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:57:17.426445 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:57:17.425902 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 13:57:17.426445 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:57:17.425926 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 13:57:17.426445 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:57:17.425929 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 13:57:17.426445 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:57:17.425949 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-958d69fff-j2glx: secret "image-registry-tls" not found Jul 09 13:57:17.426445 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:57:17.425986 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls podName:92ccdc79-fdc6-48fe-a976-23b6362217c1 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:21.425958655 +0000 UTC m=+165.279910123 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls") pod "dns-default-gnkr9" (UID: "92ccdc79-fdc6-48fe-a976-23b6362217c1") : secret "dns-default-metrics-tls" not found Jul 09 13:57:17.426445 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:57:17.426009 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls podName:0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:21.425998113 +0000 UTC m=+165.279949580 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls") pod "image-registry-958d69fff-j2glx" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2") : secret "image-registry-tls" not found Jul 09 13:57:17.426445 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:57:17.426027 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert podName:e836fc41-ecd7-4b4d-83c1-55f05ec601bd nodeName:}" failed. No retries permitted until 2026-07-09 13:58:21.426018298 +0000 UTC m=+165.279969756 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cngfc" (UID: "e836fc41-ecd7-4b4d-83c1-55f05ec601bd") : secret "networking-console-plugin-cert" not found Jul 09 13:57:18.913081 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:57:18.913049 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-slr49" Jul 09 13:57:46.439434 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:57:46.439383 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:57:46.439954 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:57:46.439557 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 13:57:46.439954 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:57:46.439631 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:48.43961399 +0000 UTC m=+252.293565443 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : secret "metrics-daemon-secret" not found Jul 09 13:58:16.551166 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:16.551122 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-958d69fff-j2glx" podUID="0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" Jul 09 13:58:16.559406 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:16.559380 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" podUID="e836fc41-ecd7-4b4d-83c1-55f05ec601bd" Jul 09 13:58:16.566649 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:16.566591 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gnkr9" podUID="92ccdc79-fdc6-48fe-a976-23b6362217c1" Jul 09 13:58:17.119589 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:58:17.119558 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:58:17.119589 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:58:17.119585 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnkr9" Jul 09 13:58:17.119894 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:58:17.119697 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:58:17.650865 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:17.650804 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 13:58:21.495058 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:58:21.495003 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 13:58:21.495058 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:58:21.495059 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 13:58:21.495528 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:58:21.495087 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 13:58:21.495528 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:21.495168 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 13:58:21.495528 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:21.495189 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 13:58:21.495528 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:21.495205 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 13:58:21.495528 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:21.495224 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-958d69fff-j2glx: secret "image-registry-tls" not found Jul 09 13:58:21.495752 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:21.495540 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls podName:0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:23.495513277 +0000 UTC m=+287.349464752 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls") pod "image-registry-958d69fff-j2glx" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2") : secret "image-registry-tls" not found Jul 09 13:58:21.495752 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:21.495580 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert podName:e836fc41-ecd7-4b4d-83c1-55f05ec601bd nodeName:}" failed. No retries permitted until 2026-07-09 14:00:23.495558794 +0000 UTC m=+287.349510249 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cngfc" (UID: "e836fc41-ecd7-4b4d-83c1-55f05ec601bd") : secret "networking-console-plugin-cert" not found Jul 09 13:58:21.496311 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:58:21.496291 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls podName:92ccdc79-fdc6-48fe-a976-23b6362217c1 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:23.496268015 +0000 UTC m=+287.350219473 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls") pod "dns-default-gnkr9" (UID: "92ccdc79-fdc6-48fe-a976-23b6362217c1") : secret "dns-default-metrics-tls" not found Jul 09 13:58:28.631867 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:58:28.631752 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:59:48.539509 ip-10-0-128-190 kubenswrapper[2575]: I0709 13:59:48.539471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 13:59:48.540044 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:59:48.539645 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 13:59:48.540044 ip-10-0-128-190 kubenswrapper[2575]: E0709 13:59:48.539717 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs podName:4e1ac675-9682-4468-993d-3d5183940a54 nodeName:}" failed. No retries permitted until 2026-07-09 14:01:50.539699643 +0000 UTC m=+374.393651097 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs") pod "network-metrics-daemon-t2rdv" (UID: "4e1ac675-9682-4468-993d-3d5183940a54") : secret "metrics-daemon-secret" not found Jul 09 14:00:00.393187 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.393101 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz"] Jul 09 14:00:00.395188 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.395168 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz" Jul 09 14:00:00.398439 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.398417 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:00:00.399652 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.399627 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4gvtl\"" Jul 09 14:00:00.399765 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.399722 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 14:00:00.400993 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.400969 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6dcfb69f8f-b6xsv"] Jul 09 14:00:00.402913 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.402894 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl"] Jul 09 14:00:00.403044 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.403026 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.404697 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.404680 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65"] Jul 09 14:00:00.404809 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.404793 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:00.406460 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.406441 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.406541 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.406477 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 09 14:00:00.406803 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.406784 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 09 14:00:00.406923 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.406839 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 14:00:00.406923 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.406882 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 14:00:00.407044 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.406867 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 09 14:00:00.407044 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.406936 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-srxlm\"" Jul 09 14:00:00.407044 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.406870 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 09 14:00:00.410155 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.410103 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 14:00:00.410155 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.410115 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 14:00:00.411002 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.410985 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2j9g9\"" Jul 09 14:00:00.411415 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.411397 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 14:00:00.411832 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.411813 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:00:00.411952 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.411824 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 14:00:00.412020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.411963 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 14:00:00.412114 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.411838 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 14:00:00.412114 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.411908 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-z8pss\"" Jul 09 14:00:00.412277 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.411842 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 14:00:00.413249 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.413229 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz"] Jul 09 14:00:00.419607 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.419583 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65"] Jul 09 14:00:00.420402 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.420381 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl"] Jul 09 14:00:00.428583 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.428562 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6dcfb69f8f-b6xsv"] Jul 09 14:00:00.527404 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527370 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:00.527580 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527418 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/178e8cc7-9a8b-4793-99da-e3af1543c23d-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-tvd65\" (UID: \"178e8cc7-9a8b-4793-99da-e3af1543c23d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.527580 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9520fb30-076b-49c3-90d6-5d6b8f87285d-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:00.527580 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527465 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.527580 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527524 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wq6r\" (UniqueName: \"kubernetes.io/projected/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-kube-api-access-7wq6r\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.527580 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527570 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfbxt\" (UniqueName: \"kubernetes.io/projected/9520fb30-076b-49c3-90d6-5d6b8f87285d-kube-api-access-bfbxt\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:00.527815 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527624 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-default-certificate\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.527815 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527678 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg2jv\" (UniqueName: \"kubernetes.io/projected/9a3f5b05-7f0a-4075-a4da-78796a325d8a-kube-api-access-pg2jv\") pod \"volume-data-source-validator-5d456fdbcd-f25wz\" (UID: \"9a3f5b05-7f0a-4075-a4da-78796a325d8a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz" Jul 09 14:00:00.527815 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527694 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.527815 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527746 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/178e8cc7-9a8b-4793-99da-e3af1543c23d-config\") pod \"service-ca-operator-5d46d9ffd4-tvd65\" (UID: \"178e8cc7-9a8b-4793-99da-e3af1543c23d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.527815 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527811 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-stats-auth\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.528129 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.527876 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6tnj\" (UniqueName: \"kubernetes.io/projected/178e8cc7-9a8b-4793-99da-e3af1543c23d-kube-api-access-k6tnj\") pod \"service-ca-operator-5d46d9ffd4-tvd65\" (UID: \"178e8cc7-9a8b-4793-99da-e3af1543c23d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.628954 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.628904 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfbxt\" (UniqueName: \"kubernetes.io/projected/9520fb30-076b-49c3-90d6-5d6b8f87285d-kube-api-access-bfbxt\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:00.629163 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.628974 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-default-certificate\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.629163 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629028 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg2jv\" (UniqueName: \"kubernetes.io/projected/9a3f5b05-7f0a-4075-a4da-78796a325d8a-kube-api-access-pg2jv\") pod \"volume-data-source-validator-5d456fdbcd-f25wz\" (UID: \"9a3f5b05-7f0a-4075-a4da-78796a325d8a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz" Jul 09 14:00:00.629163 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629054 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.629163 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629080 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/178e8cc7-9a8b-4793-99da-e3af1543c23d-config\") pod \"service-ca-operator-5d46d9ffd4-tvd65\" (UID: \"178e8cc7-9a8b-4793-99da-e3af1543c23d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.629372 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:00.629157 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:00:00.629372 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629220 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-stats-auth\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.629372 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:00.629232 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:01.129215015 +0000 UTC m=+264.983166482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : secret "router-metrics-certs-default" not found Jul 09 14:00:00.629372 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6tnj\" (UniqueName: \"kubernetes.io/projected/178e8cc7-9a8b-4793-99da-e3af1543c23d-kube-api-access-k6tnj\") pod \"service-ca-operator-5d46d9ffd4-tvd65\" (UID: \"178e8cc7-9a8b-4793-99da-e3af1543c23d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.629372 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629306 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:00.629619 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629373 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/178e8cc7-9a8b-4793-99da-e3af1543c23d-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-tvd65\" (UID: \"178e8cc7-9a8b-4793-99da-e3af1543c23d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.629619 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629402 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9520fb30-076b-49c3-90d6-5d6b8f87285d-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:00.629619 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629439 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.629619 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:00.629452 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:00.629619 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629473 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wq6r\" (UniqueName: \"kubernetes.io/projected/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-kube-api-access-7wq6r\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.629619 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:00.629503 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls podName:9520fb30-076b-49c3-90d6-5d6b8f87285d nodeName:}" failed. No retries permitted until 2026-07-09 14:00:01.129484627 +0000 UTC m=+264.983436082 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5f6fl" (UID: "9520fb30-076b-49c3-90d6-5d6b8f87285d") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:00.629911 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:00.629633 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:01.129621191 +0000 UTC m=+264.983572677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : configmap references non-existent config key: service-ca.crt Jul 09 14:00:00.629911 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.629732 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/178e8cc7-9a8b-4793-99da-e3af1543c23d-config\") pod \"service-ca-operator-5d46d9ffd4-tvd65\" (UID: \"178e8cc7-9a8b-4793-99da-e3af1543c23d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.630448 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.630421 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9520fb30-076b-49c3-90d6-5d6b8f87285d-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:00.631704 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.631676 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/178e8cc7-9a8b-4793-99da-e3af1543c23d-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-tvd65\" (UID: \"178e8cc7-9a8b-4793-99da-e3af1543c23d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.631937 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.631918 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-default-certificate\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.631984 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.631923 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-stats-auth\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.640342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.640310 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg2jv\" (UniqueName: \"kubernetes.io/projected/9a3f5b05-7f0a-4075-a4da-78796a325d8a-kube-api-access-pg2jv\") pod \"volume-data-source-validator-5d456fdbcd-f25wz\" (UID: \"9a3f5b05-7f0a-4075-a4da-78796a325d8a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz" Jul 09 14:00:00.640566 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.640546 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6tnj\" (UniqueName: \"kubernetes.io/projected/178e8cc7-9a8b-4793-99da-e3af1543c23d-kube-api-access-k6tnj\") pod \"service-ca-operator-5d46d9ffd4-tvd65\" (UID: \"178e8cc7-9a8b-4793-99da-e3af1543c23d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.640657 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.640623 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wq6r\" (UniqueName: \"kubernetes.io/projected/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-kube-api-access-7wq6r\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:00.640897 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.640878 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfbxt\" (UniqueName: \"kubernetes.io/projected/9520fb30-076b-49c3-90d6-5d6b8f87285d-kube-api-access-bfbxt\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:00.704970 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.704837 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz" Jul 09 14:00:00.727528 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.727493 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" Jul 09 14:00:00.844069 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.844042 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz"] Jul 09 14:00:00.846951 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:00.846915 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a3f5b05_7f0a_4075_a4da_78796a325d8a.slice/crio-5bb2fdf648345ea543f3fd22ea9eebe6f6ff209f9e38cb7560e92490af563e64 WatchSource:0}: Error finding container 5bb2fdf648345ea543f3fd22ea9eebe6f6ff209f9e38cb7560e92490af563e64: Status 404 returned error can't find the container with id 5bb2fdf648345ea543f3fd22ea9eebe6f6ff209f9e38cb7560e92490af563e64 Jul 09 14:00:00.865164 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:00.865131 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65"] Jul 09 14:00:00.868565 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:00.868541 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod178e8cc7_9a8b_4793_99da_e3af1543c23d.slice/crio-7b2dfb461a3f643302f8ecb823c3c8974a1de40cd1af3d07584ef493abeff117 WatchSource:0}: Error finding container 7b2dfb461a3f643302f8ecb823c3c8974a1de40cd1af3d07584ef493abeff117: Status 404 returned error can't find the container with id 7b2dfb461a3f643302f8ecb823c3c8974a1de40cd1af3d07584ef493abeff117 Jul 09 14:00:01.134574 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:01.134534 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:01.134777 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:01.134595 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:01.134777 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:01.134651 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:01.134777 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:01.134699 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:01.134777 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:01.134746 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:00:01.134777 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:01.134767 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls podName:9520fb30-076b-49c3-90d6-5d6b8f87285d nodeName:}" failed. No retries permitted until 2026-07-09 14:00:02.134752592 +0000 UTC m=+265.988704046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5f6fl" (UID: "9520fb30-076b-49c3-90d6-5d6b8f87285d") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:01.135049 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:01.134791 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:02.134773031 +0000 UTC m=+265.988724485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : configmap references non-existent config key: service-ca.crt Jul 09 14:00:01.135049 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:01.134817 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:02.134807317 +0000 UTC m=+265.988758771 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : secret "router-metrics-certs-default" not found Jul 09 14:00:01.365705 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:01.365669 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" event={"ID":"178e8cc7-9a8b-4793-99da-e3af1543c23d","Type":"ContainerStarted","Data":"7b2dfb461a3f643302f8ecb823c3c8974a1de40cd1af3d07584ef493abeff117"} Jul 09 14:00:01.366786 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:01.366761 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz" event={"ID":"9a3f5b05-7f0a-4075-a4da-78796a325d8a","Type":"ContainerStarted","Data":"5bb2fdf648345ea543f3fd22ea9eebe6f6ff209f9e38cb7560e92490af563e64"} Jul 09 14:00:02.142928 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:02.142870 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:02.143367 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:02.142980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:02.143367 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:02.143040 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:02.143367 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:02.143056 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:00:02.143367 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:02.143148 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:02.143367 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:02.143155 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:04.143130807 +0000 UTC m=+267.997082266 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : secret "router-metrics-certs-default" not found Jul 09 14:00:02.143367 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:02.143228 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:04.143209162 +0000 UTC m=+267.997160624 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : configmap references non-existent config key: service-ca.crt Jul 09 14:00:02.143367 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:02.143246 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls podName:9520fb30-076b-49c3-90d6-5d6b8f87285d nodeName:}" failed. No retries permitted until 2026-07-09 14:00:04.143236827 +0000 UTC m=+267.997188282 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5f6fl" (UID: "9520fb30-076b-49c3-90d6-5d6b8f87285d") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:02.371613 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:02.371496 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz" event={"ID":"9a3f5b05-7f0a-4075-a4da-78796a325d8a","Type":"ContainerStarted","Data":"dad56d7ecc00b7d00cedd71941a137990559c511980014e3975f687c36cd2945"} Jul 09 14:00:02.392655 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:02.392592 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-f25wz" podStartSLOduration=1.14089752 podStartE2EDuration="2.392569546s" totalImagesPullingTime="1.251672026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:02.391602078 +0000 UTC m=+266.245553555" watchObservedRunningTime="2026-07-09 14:00:02.392569546 +0000 UTC m=+266.246521023" Jul 09 14:00:04.159773 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:04.159731 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:04.160262 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:04.159796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:04.160262 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:04.159873 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:04.160262 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:04.159943 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:00:04.160262 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:04.160015 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:08.160000099 +0000 UTC m=+272.013951554 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : secret "router-metrics-certs-default" not found Jul 09 14:00:04.160262 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:04.159950 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:04.160262 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:04.160030 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:08.160023664 +0000 UTC m=+272.013975118 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : configmap references non-existent config key: service-ca.crt Jul 09 14:00:04.160262 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:04.160063 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls podName:9520fb30-076b-49c3-90d6-5d6b8f87285d nodeName:}" failed. No retries permitted until 2026-07-09 14:00:08.160044147 +0000 UTC m=+272.013995606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5f6fl" (UID: "9520fb30-076b-49c3-90d6-5d6b8f87285d") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:04.377635 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:04.377596 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" event={"ID":"178e8cc7-9a8b-4793-99da-e3af1543c23d","Type":"ContainerStarted","Data":"b736c40337672d85ef4ef7273d8550981d80fe614ff4dd5f07ec131ae72b771b"} Jul 09 14:00:04.396101 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:04.396043 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" podStartSLOduration=1.863852012 podStartE2EDuration="4.396022262s" totalImagesPullingTime="2.53217025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:04.395598496 +0000 UTC m=+268.249549972" watchObservedRunningTime="2026-07-09 14:00:04.396022262 +0000 UTC m=+268.249973759" Jul 09 14:00:05.655463 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.655419 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9hskn"] Jul 09 14:00:05.657918 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.657894 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.662844 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.662820 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 14:00:05.663001 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.662844 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 14:00:05.663001 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.662869 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 14:00:05.663970 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.663942 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 14:00:05.664092 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.663949 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kq44v\"" Jul 09 14:00:05.672739 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.672708 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9hskn"] Jul 09 14:00:05.773992 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.773956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-crio-socket\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.774162 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.774002 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.774162 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.774107 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77kj8\" (UniqueName: \"kubernetes.io/projected/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-api-access-77kj8\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.774233 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.774172 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-data-volume\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.774271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.774257 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.874622 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.874577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-crio-socket\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.874622 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.874629 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.874928 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:05.874695 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:00:05.874928 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.874712 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-crio-socket\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.874928 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:05.874767 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:06.374745019 +0000 UTC m=+270.228696478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : configmap "kube-rbac-proxy" not found Jul 09 14:00:05.874928 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.874795 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77kj8\" (UniqueName: \"kubernetes.io/projected/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-api-access-77kj8\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.874928 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.874909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-data-volume\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.875162 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.875007 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.875162 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:05.875079 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:00:05.875162 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:05.875121 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:06.375109392 +0000 UTC m=+270.229060846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : secret "insights-runtime-extractor-tls" not found Jul 09 14:00:05.875270 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.875249 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-data-volume\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:05.884697 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:05.884665 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77kj8\" (UniqueName: \"kubernetes.io/projected/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-api-access-77kj8\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:06.379022 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.378980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:06.379241 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.379051 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:06.379241 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:06.379132 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:00:06.379241 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:06.379138 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:00:06.379241 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:06.379208 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:07.379186693 +0000 UTC m=+271.233138147 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : configmap "kube-rbac-proxy" not found Jul 09 14:00:06.379241 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:06.379234 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:07.379215517 +0000 UTC m=+271.233166987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : secret "insights-runtime-extractor-tls" not found Jul 09 14:00:06.625167 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.625130 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-fmg9k"] Jul 09 14:00:06.627531 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.627502 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.630685 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.630625 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 14:00:06.630685 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.630634 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 14:00:06.631965 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.631940 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-42mgs\"" Jul 09 14:00:06.632279 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.632243 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 14:00:06.632381 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.632352 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 14:00:06.640928 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.640895 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-fmg9k"] Jul 09 14:00:06.782442 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.782403 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf-signing-cabundle\") pod \"service-ca-758f599c5-fmg9k\" (UID: \"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf\") " pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.782442 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.782445 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf-signing-key\") pod \"service-ca-758f599c5-fmg9k\" (UID: \"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf\") " pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.782881 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.782536 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsvbt\" (UniqueName: \"kubernetes.io/projected/a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf-kube-api-access-lsvbt\") pod \"service-ca-758f599c5-fmg9k\" (UID: \"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf\") " pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.883091 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.882999 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsvbt\" (UniqueName: \"kubernetes.io/projected/a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf-kube-api-access-lsvbt\") pod \"service-ca-758f599c5-fmg9k\" (UID: \"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf\") " pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.883091 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.883095 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf-signing-cabundle\") pod \"service-ca-758f599c5-fmg9k\" (UID: \"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf\") " pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.883284 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.883114 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf-signing-key\") pod \"service-ca-758f599c5-fmg9k\" (UID: \"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf\") " pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.883866 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.883825 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf-signing-cabundle\") pod \"service-ca-758f599c5-fmg9k\" (UID: \"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf\") " pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.885611 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.885588 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf-signing-key\") pod \"service-ca-758f599c5-fmg9k\" (UID: \"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf\") " pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.894292 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.894270 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsvbt\" (UniqueName: \"kubernetes.io/projected/a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf-kube-api-access-lsvbt\") pod \"service-ca-758f599c5-fmg9k\" (UID: \"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf\") " pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:06.942196 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:06.942159 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-fmg9k" Jul 09 14:00:07.069157 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:07.069128 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-fmg9k"] Jul 09 14:00:07.071485 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:07.071451 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57bbd5d_c3a0_42c0_9ba1_5b6fa189f9bf.slice/crio-0268a36d8cd79d6a078303bb1bcaa5ae294951287094fe84a966b62ce496fe3f WatchSource:0}: Error finding container 0268a36d8cd79d6a078303bb1bcaa5ae294951287094fe84a966b62ce496fe3f: Status 404 returned error can't find the container with id 0268a36d8cd79d6a078303bb1bcaa5ae294951287094fe84a966b62ce496fe3f Jul 09 14:00:07.386066 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:07.386034 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:07.386288 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:07.386104 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:07.386288 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:07.386184 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:00:07.386288 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:07.386189 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:00:07.386288 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:07.386250 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:09.386234392 +0000 UTC m=+273.240185846 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : secret "insights-runtime-extractor-tls" not found Jul 09 14:00:07.386288 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:07.386271 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:09.386259672 +0000 UTC m=+273.240211126 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : configmap "kube-rbac-proxy" not found Jul 09 14:00:07.388151 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:07.388118 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-fmg9k" event={"ID":"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf","Type":"ContainerStarted","Data":"c640e32fcb5648fe19104d5fb812e7c77b109cdaff239e2231c43f9dbfc5d590"} Jul 09 14:00:07.388295 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:07.388164 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-fmg9k" event={"ID":"a57bbd5d-c3a0-42c0-9ba1-5b6fa189f9bf","Type":"ContainerStarted","Data":"0268a36d8cd79d6a078303bb1bcaa5ae294951287094fe84a966b62ce496fe3f"} Jul 09 14:00:07.412927 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:07.412839 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-fmg9k" podStartSLOduration=1.412822479 podStartE2EDuration="1.412822479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:07.412213697 +0000 UTC m=+271.266165173" watchObservedRunningTime="2026-07-09 14:00:07.412822479 +0000 UTC m=+271.266773955" Jul 09 14:00:08.192975 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:08.192924 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:08.193434 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:08.193074 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:08.193434 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:08.193093 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:16.193076565 +0000 UTC m=+280.047028018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : configmap references non-existent config key: service-ca.crt Jul 09 14:00:08.193434 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:08.193164 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:08.193434 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:08.193180 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 14:00:08.193434 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:08.193232 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs podName:2f13c2b9-d38e-4af1-b7bd-3caf13d73f57 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:16.193215819 +0000 UTC m=+280.047167320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs") pod "router-default-6dcfb69f8f-b6xsv" (UID: "2f13c2b9-d38e-4af1-b7bd-3caf13d73f57") : secret "router-metrics-certs-default" not found Jul 09 14:00:08.193434 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:08.193269 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:08.193434 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:08.193305 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls podName:9520fb30-076b-49c3-90d6-5d6b8f87285d nodeName:}" failed. No retries permitted until 2026-07-09 14:00:16.19329499 +0000 UTC m=+280.047246458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-5f6fl" (UID: "9520fb30-076b-49c3-90d6-5d6b8f87285d") : secret "cluster-monitoring-operator-tls" not found Jul 09 14:00:08.648694 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:08.648660 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xd872_45e4c2e5-2a9c-4508-a9d1-e2c274713331/dns-node-resolver/0.log" Jul 09 14:00:09.247054 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:09.247029 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5tgvd_232c0b60-887d-41bd-8563-67bd303f57e4/node-ca/0.log" Jul 09 14:00:09.403901 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:09.403865 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:09.404084 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:09.403919 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:09.404084 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:09.404041 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:00:09.404167 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:09.404046 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:00:09.404167 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:09.404121 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:13.404102959 +0000 UTC m=+277.258054413 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : secret "insights-runtime-extractor-tls" not found Jul 09 14:00:09.404248 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:09.404184 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:13.404170307 +0000 UTC m=+277.258121762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : configmap "kube-rbac-proxy" not found Jul 09 14:00:10.247807 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:10.247763 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5xqw6_48bc75ef-a454-4f16-8861-a3ca613afd68/serve-healthcheck-canary/0.log" Jul 09 14:00:13.439298 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:13.439257 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:13.439687 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:13.439310 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:13.439687 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:13.439388 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 14:00:13.439687 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:13.439419 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 14:00:13.439687 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:13.439439 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:21.439426039 +0000 UTC m=+285.293377493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : configmap "kube-rbac-proxy" not found Jul 09 14:00:13.439687 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:13.439475 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls podName:f6559b7d-0f35-49e5-a64b-b43d2c566bfe nodeName:}" failed. No retries permitted until 2026-07-09 14:00:21.439460937 +0000 UTC m=+285.293412391 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9hskn" (UID: "f6559b7d-0f35-49e5-a64b-b43d2c566bfe") : secret "insights-runtime-extractor-tls" not found Jul 09 14:00:16.266750 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.266699 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:16.267259 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.266828 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:16.267259 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.266894 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:16.267415 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.267391 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-service-ca-bundle\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:16.269506 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.269483 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f13c2b9-d38e-4af1-b7bd-3caf13d73f57-metrics-certs\") pod \"router-default-6dcfb69f8f-b6xsv\" (UID: \"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57\") " pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:16.269617 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.269593 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9520fb30-076b-49c3-90d6-5d6b8f87285d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-5f6fl\" (UID: \"9520fb30-076b-49c3-90d6-5d6b8f87285d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:16.313618 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.313572 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:16.321492 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.321460 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" Jul 09 14:00:16.467340 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.467308 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6dcfb69f8f-b6xsv"] Jul 09 14:00:16.469869 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:16.469823 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f13c2b9_d38e_4af1_b7bd_3caf13d73f57.slice/crio-e11e2e11312ce7785f041ddd29b9d7ddd727ed4ff62376def31f0435fa6a2df0 WatchSource:0}: Error finding container e11e2e11312ce7785f041ddd29b9d7ddd727ed4ff62376def31f0435fa6a2df0: Status 404 returned error can't find the container with id e11e2e11312ce7785f041ddd29b9d7ddd727ed4ff62376def31f0435fa6a2df0 Jul 09 14:00:16.482910 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:16.482789 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl"] Jul 09 14:00:16.485747 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:16.485716 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9520fb30_076b_49c3_90d6_5d6b8f87285d.slice/crio-4561f05771e7b158171f7e04b188cbffcd3f6c781c2437bd701afd713392ddab WatchSource:0}: Error finding container 4561f05771e7b158171f7e04b188cbffcd3f6c781c2437bd701afd713392ddab: Status 404 returned error can't find the container with id 4561f05771e7b158171f7e04b188cbffcd3f6c781c2437bd701afd713392ddab Jul 09 14:00:17.422273 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:17.422233 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" event={"ID":"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57","Type":"ContainerStarted","Data":"d4130a677c0f6aa5bc0ecfac092f01cca7d3417690ca64b38c455920daf381c1"} Jul 09 14:00:17.422712 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:17.422288 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" event={"ID":"2f13c2b9-d38e-4af1-b7bd-3caf13d73f57","Type":"ContainerStarted","Data":"e11e2e11312ce7785f041ddd29b9d7ddd727ed4ff62376def31f0435fa6a2df0"} Jul 09 14:00:17.423423 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:17.423389 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" event={"ID":"9520fb30-076b-49c3-90d6-5d6b8f87285d","Type":"ContainerStarted","Data":"4561f05771e7b158171f7e04b188cbffcd3f6c781c2437bd701afd713392ddab"} Jul 09 14:00:17.442536 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:17.442483 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" podStartSLOduration=17.442467769 podStartE2EDuration="17.442467769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:17.441557301 +0000 UTC m=+281.295508777" watchObservedRunningTime="2026-07-09 14:00:17.442467769 +0000 UTC m=+281.296419245" Jul 09 14:00:18.314168 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:18.314130 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:18.316906 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:18.316877 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:18.427255 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:18.427225 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:18.428737 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:18.428712 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6dcfb69f8f-b6xsv" Jul 09 14:00:20.121020 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:20.120978 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gnkr9" podUID="92ccdc79-fdc6-48fe-a976-23b6362217c1" Jul 09 14:00:20.121020 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:20.121006 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-958d69fff-j2glx" podUID="0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" Jul 09 14:00:20.121542 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:20.120988 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" podUID="e836fc41-ecd7-4b4d-83c1-55f05ec601bd" Jul 09 14:00:20.435544 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:20.435449 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 14:00:20.435544 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:20.435448 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" event={"ID":"9520fb30-076b-49c3-90d6-5d6b8f87285d","Type":"ContainerStarted","Data":"485afa9ee685b924cf352bd048819985280b3860b1c382e1d3b6127388b7388c"} Jul 09 14:00:20.435544 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:20.435497 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 14:00:20.435544 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:20.435514 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnkr9" Jul 09 14:00:20.464392 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:20.464336 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-5f6fl" podStartSLOduration=17.354578359 podStartE2EDuration="20.464318656s" totalImagesPullingTime="3.109740297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:20.464120893 +0000 UTC m=+284.318072370" watchObservedRunningTime="2026-07-09 14:00:20.464318656 +0000 UTC m=+284.318270159" Jul 09 14:00:21.511940 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:21.511900 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:21.512384 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:21.512043 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:21.512576 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:21.512553 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:21.514585 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:21.514564 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f6559b7d-0f35-49e5-a64b-b43d2c566bfe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9hskn\" (UID: \"f6559b7d-0f35-49e5-a64b-b43d2c566bfe\") " pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:21.567004 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:21.566969 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9hskn" Jul 09 14:00:21.701742 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:21.701579 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9hskn"] Jul 09 14:00:21.704731 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:21.704681 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6559b7d_0f35_49e5_a64b_b43d2c566bfe.slice/crio-3bcc930069c81cf1ea9f8b5f3f900419d0d81a5f5c7db8630a248ce8365f3656 WatchSource:0}: Error finding container 3bcc930069c81cf1ea9f8b5f3f900419d0d81a5f5c7db8630a248ce8365f3656: Status 404 returned error can't find the container with id 3bcc930069c81cf1ea9f8b5f3f900419d0d81a5f5c7db8630a248ce8365f3656 Jul 09 14:00:22.442706 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:22.442669 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hskn" event={"ID":"f6559b7d-0f35-49e5-a64b-b43d2c566bfe","Type":"ContainerStarted","Data":"3bcc930069c81cf1ea9f8b5f3f900419d0d81a5f5c7db8630a248ce8365f3656"} Jul 09 14:00:23.530524 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.530480 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 14:00:23.531069 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.530577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 14:00:23.531069 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.530601 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 14:00:23.533202 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.533173 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/92ccdc79-fdc6-48fe-a976-23b6362217c1-metrics-tls\") pod \"dns-default-gnkr9\" (UID: \"92ccdc79-fdc6-48fe-a976-23b6362217c1\") " pod="openshift-dns/dns-default-gnkr9" Jul 09 14:00:23.533377 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.533352 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"image-registry-958d69fff-j2glx\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 14:00:23.533419 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.533388 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e836fc41-ecd7-4b4d-83c1-55f05ec601bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cngfc\" (UID: \"e836fc41-ecd7-4b4d-83c1-55f05ec601bd\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 14:00:23.740322 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.740282 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dkvmm\"" Jul 09 14:00:23.740517 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.740345 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6tr79\"" Jul 09 14:00:23.740517 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.740287 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-blxcx\"" Jul 09 14:00:23.747115 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.747074 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnkr9" Jul 09 14:00:23.747286 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.747123 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 14:00:23.747286 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:23.747145 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" Jul 09 14:00:25.876677 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:25.876586 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gnkr9"] Jul 09 14:00:25.880159 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:25.880118 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92ccdc79_fdc6_48fe_a976_23b6362217c1.slice/crio-bb1ff99e8cd8e2fc9ab3ac14c33eea47171e3480ffcdd4a1822353a52bbe1922 WatchSource:0}: Error finding container bb1ff99e8cd8e2fc9ab3ac14c33eea47171e3480ffcdd4a1822353a52bbe1922: Status 404 returned error can't find the container with id bb1ff99e8cd8e2fc9ab3ac14c33eea47171e3480ffcdd4a1822353a52bbe1922 Jul 09 14:00:25.897673 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:25.897637 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-cngfc"] Jul 09 14:00:25.901389 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:25.901358 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-958d69fff-j2glx"] Jul 09 14:00:25.901565 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:25.901540 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode836fc41_ecd7_4b4d_83c1_55f05ec601bd.slice/crio-11816c01f130f51b4ded0ad9dd08ea5ed610017dc5771df6c9abdaeb0e51b6c0 WatchSource:0}: Error finding container 11816c01f130f51b4ded0ad9dd08ea5ed610017dc5771df6c9abdaeb0e51b6c0: Status 404 returned error can't find the container with id 11816c01f130f51b4ded0ad9dd08ea5ed610017dc5771df6c9abdaeb0e51b6c0 Jul 09 14:00:25.904778 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:25.904750 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c07a4c8_f329_4a1e_a5e0_36b0d4461cd2.slice/crio-7d3a1762eb978c86751493839dea9b8bf5e9d24a109ee0afd55215da6fed6fc3 WatchSource:0}: Error finding container 7d3a1762eb978c86751493839dea9b8bf5e9d24a109ee0afd55215da6fed6fc3: Status 404 returned error can't find the container with id 7d3a1762eb978c86751493839dea9b8bf5e9d24a109ee0afd55215da6fed6fc3 Jul 09 14:00:26.457922 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:26.457886 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-958d69fff-j2glx" event={"ID":"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2","Type":"ContainerStarted","Data":"35546c15239addfb7e78cee7a64ab97f8287f9bd1a5eb6fe89fcb99e95f41b33"} Jul 09 14:00:26.458119 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:26.457941 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-958d69fff-j2glx" event={"ID":"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2","Type":"ContainerStarted","Data":"7d3a1762eb978c86751493839dea9b8bf5e9d24a109ee0afd55215da6fed6fc3"} Jul 09 14:00:26.458119 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:26.458016 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 14:00:26.459719 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:26.459694 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hskn" event={"ID":"f6559b7d-0f35-49e5-a64b-b43d2c566bfe","Type":"ContainerStarted","Data":"915f43e34e106f7ffe341b001f870e9df3a3c218c15bd7312a22497c500d5628"} Jul 09 14:00:26.459867 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:26.459740 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hskn" event={"ID":"f6559b7d-0f35-49e5-a64b-b43d2c566bfe","Type":"ContainerStarted","Data":"feca0f9778c0c692f0dc4d0a33f376c5ba33fc3b0f58062a02010b5552076642"} Jul 09 14:00:26.460843 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:26.460825 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" event={"ID":"e836fc41-ecd7-4b4d-83c1-55f05ec601bd","Type":"ContainerStarted","Data":"11816c01f130f51b4ded0ad9dd08ea5ed610017dc5771df6c9abdaeb0e51b6c0"} Jul 09 14:00:26.461794 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:26.461778 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnkr9" event={"ID":"92ccdc79-fdc6-48fe-a976-23b6362217c1","Type":"ContainerStarted","Data":"bb1ff99e8cd8e2fc9ab3ac14c33eea47171e3480ffcdd4a1822353a52bbe1922"} Jul 09 14:00:26.482751 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:26.482692 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-958d69fff-j2glx" podStartSLOduration=265.482675997 podStartE2EDuration="4m25.482675997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:26.481280959 +0000 UTC m=+290.335232435" watchObservedRunningTime="2026-07-09 14:00:26.482675997 +0000 UTC m=+290.336627452" Jul 09 14:00:28.472083 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:28.472039 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnkr9" event={"ID":"92ccdc79-fdc6-48fe-a976-23b6362217c1","Type":"ContainerStarted","Data":"81c5497495327e105d9673cc69377c51ef6badf58fae0176dabc28aa199eeec3"} Jul 09 14:00:28.472502 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:28.472104 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnkr9" event={"ID":"92ccdc79-fdc6-48fe-a976-23b6362217c1","Type":"ContainerStarted","Data":"78e9f8261b2a664e5b56d52e40fe50bae36fbc4e44ea8a7700399f9702bd12d8"} Jul 09 14:00:28.472502 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:28.472178 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gnkr9" Jul 09 14:00:28.491043 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:28.490983 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gnkr9" podStartSLOduration=253.427227375 podStartE2EDuration="4m15.490964193s" totalImagesPullingTime="2.063736818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:28.490422154 +0000 UTC m=+292.344373631" watchObservedRunningTime="2026-07-09 14:00:28.490964193 +0000 UTC m=+292.344915670" Jul 09 14:00:30.481912 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:30.481787 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9hskn" event={"ID":"f6559b7d-0f35-49e5-a64b-b43d2c566bfe","Type":"ContainerStarted","Data":"ae4a4e3135e774af7904bb57041cf0fa361e14de88576b2869068b73810d2fca"} Jul 09 14:00:30.483355 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:30.483329 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" event={"ID":"e836fc41-ecd7-4b4d-83c1-55f05ec601bd","Type":"ContainerStarted","Data":"3b09e7948e3eba404b5e8dcb1105144e999731c98ea5292efee8d4d738e4772d"} Jul 09 14:00:30.502287 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:30.502223 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9hskn" podStartSLOduration=18.030429518 podStartE2EDuration="25.502200667s" totalImagesPullingTime="7.471771149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:00:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:30.501890122 +0000 UTC m=+294.355841590" watchObservedRunningTime="2026-07-09 14:00:30.502200667 +0000 UTC m=+294.356152143" Jul 09 14:00:30.524818 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:30.524757 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cngfc" podStartSLOduration=266.771075111 podStartE2EDuration="4m30.524738771s" totalImagesPullingTime="3.75366366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:30.524248676 +0000 UTC m=+294.378200150" watchObservedRunningTime="2026-07-09 14:00:30.524738771 +0000 UTC m=+294.378690247" Jul 09 14:00:31.632642 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:31.632593 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-t2rdv" podUID="4e1ac675-9682-4468-993d-3d5183940a54" Jul 09 14:00:35.172682 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.172643 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz"] Jul 09 14:00:35.177507 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.177486 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.181376 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.181343 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 14:00:35.181524 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.181377 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rjzmj\"" Jul 09 14:00:35.182385 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.182361 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 14:00:35.182563 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.182441 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 14:00:35.190978 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.190947 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz"] Jul 09 14:00:35.220272 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.220233 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.220462 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.220327 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.220462 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.220361 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.220462 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.220389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fjvx\" (UniqueName: \"kubernetes.io/projected/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-kube-api-access-8fjvx\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.321682 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.321639 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fjvx\" (UniqueName: \"kubernetes.io/projected/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-kube-api-access-8fjvx\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.321917 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.321754 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.321917 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.321812 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.321917 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.321838 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.322030 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:35.321942 2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 09 14:00:35.322030 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:35.322028 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-prometheus-operator-tls podName:18fa7a34-4f44-4ee7-81f5-9689e72ca51b nodeName:}" failed. No retries permitted until 2026-07-09 14:00:35.822010257 +0000 UTC m=+299.675961712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-rfvhz" (UID: "18fa7a34-4f44-4ee7-81f5-9689e72ca51b") : secret "prometheus-operator-tls" not found Jul 09 14:00:35.322518 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.322483 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.324502 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.324478 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.332619 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.332594 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fjvx\" (UniqueName: \"kubernetes.io/projected/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-kube-api-access-8fjvx\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.826468 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.826428 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:35.829093 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:35.829060 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/18fa7a34-4f44-4ee7-81f5-9689e72ca51b-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-rfvhz\" (UID: \"18fa7a34-4f44-4ee7-81f5-9689e72ca51b\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:36.086985 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:36.086898 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" Jul 09 14:00:36.241442 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:36.241409 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz"] Jul 09 14:00:36.245833 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:36.245807 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18fa7a34_4f44_4ee7_81f5_9689e72ca51b.slice/crio-90b1b15030876ff5ca884459a5e73b203f33936a79baffef063a1b93d66a1d64 WatchSource:0}: Error finding container 90b1b15030876ff5ca884459a5e73b203f33936a79baffef063a1b93d66a1d64: Status 404 returned error can't find the container with id 90b1b15030876ff5ca884459a5e73b203f33936a79baffef063a1b93d66a1d64 Jul 09 14:00:36.504370 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:36.504272 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" event={"ID":"18fa7a34-4f44-4ee7-81f5-9689e72ca51b","Type":"ContainerStarted","Data":"90b1b15030876ff5ca884459a5e73b203f33936a79baffef063a1b93d66a1d64"} Jul 09 14:00:36.548819 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:36.548790 2575 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 14:00:38.478161 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:38.478133 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gnkr9" Jul 09 14:00:38.514886 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:38.514829 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" event={"ID":"18fa7a34-4f44-4ee7-81f5-9689e72ca51b","Type":"ContainerStarted","Data":"fe052c26c9d8b7073c14c475527ed13c98c80a430a64c3d3a156d23cfca4b7a3"} Jul 09 14:00:39.520440 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:39.520400 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" event={"ID":"18fa7a34-4f44-4ee7-81f5-9689e72ca51b","Type":"ContainerStarted","Data":"a97ca5a993e944ce91518de56f993b4378977113062b4d5b9395b9545cb42dea"} Jul 09 14:00:39.538867 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:39.538799 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rfvhz" podStartSLOduration=2.410331163 podStartE2EDuration="4.53878359s" totalImagesPullingTime="2.128452427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:39.537907382 +0000 UTC m=+303.391858858" watchObservedRunningTime="2026-07-09 14:00:39.53878359 +0000 UTC m=+303.392735078" Jul 09 14:00:41.545208 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.545167 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq"] Jul 09 14:00:41.548463 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.548444 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.551224 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.551201 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 09 14:00:41.551224 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.551211 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4rjr6\"" Jul 09 14:00:41.551437 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.551287 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 09 14:00:41.559242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.558779 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh"] Jul 09 14:00:41.562419 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.562394 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.563417 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.563392 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq"] Jul 09 14:00:41.565317 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.565133 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-c7txw\"" Jul 09 14:00:41.565317 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.565141 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 14:00:41.565317 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.565146 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 14:00:41.565662 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.565640 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 14:00:41.576352 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.576311 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh"] Jul 09 14:00:41.593876 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.593816 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7bf4x"] Jul 09 14:00:41.598381 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.598313 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.601472 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.601442 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 14:00:41.601912 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.601890 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 14:00:41.602031 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.602014 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 14:00:41.602567 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.602549 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mkrgj\"" Jul 09 14:00:41.680893 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.680833 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.681068 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.680906 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fefb73d-befb-4eac-ba98-719b13989a8b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.681068 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.680953 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-wtmp\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.681068 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.680985 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svj6d\" (UniqueName: \"kubernetes.io/projected/54448918-e070-4b17-8bc3-010541c874b9-kube-api-access-svj6d\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.681068 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681011 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54448918-e070-4b17-8bc3-010541c874b9-root\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.681068 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681035 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6vpg\" (UniqueName: \"kubernetes.io/projected/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-api-access-z6vpg\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.681234 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681069 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54448918-e070-4b17-8bc3-010541c874b9-sys\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.681234 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-tls\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.681234 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681122 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-textfile\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.681234 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681153 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3be9c8d3-a431-4c72-94ab-dd15bf815990-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.681234 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681185 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.681380 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3be9c8d3-a431-4c72-94ab-dd15bf815990-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.681380 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681273 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.681380 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681300 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.681380 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be9c8d3-a431-4c72-94ab-dd15bf815990-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.681380 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681370 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5fefb73d-befb-4eac-ba98-719b13989a8b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.681523 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681405 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv6l8\" (UniqueName: \"kubernetes.io/projected/3be9c8d3-a431-4c72-94ab-dd15bf815990-kube-api-access-jv6l8\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.681523 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681459 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54448918-e070-4b17-8bc3-010541c874b9-metrics-client-ca\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.681523 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.681487 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.781896 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.781838 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svj6d\" (UniqueName: \"kubernetes.io/projected/54448918-e070-4b17-8bc3-010541c874b9-kube-api-access-svj6d\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.781896 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.781899 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54448918-e070-4b17-8bc3-010541c874b9-root\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.782158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.781929 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6vpg\" (UniqueName: \"kubernetes.io/projected/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-api-access-z6vpg\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.782158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.781960 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54448918-e070-4b17-8bc3-010541c874b9-sys\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.782158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.781987 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-tls\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.782158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782014 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-textfile\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.782158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782031 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54448918-e070-4b17-8bc3-010541c874b9-root\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.782158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782042 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3be9c8d3-a431-4c72-94ab-dd15bf815990-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.782158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782104 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.782158 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:41.782136 2575 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 09 14:00:41.782158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782150 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3be9c8d3-a431-4c72-94ab-dd15bf815990-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:41.782202 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3be9c8d3-a431-4c72-94ab-dd15bf815990-openshift-state-metrics-tls podName:3be9c8d3-a431-4c72-94ab-dd15bf815990 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:42.282178146 +0000 UTC m=+306.136129603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3be9c8d3-a431-4c72-94ab-dd15bf815990-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-jksvq" (UID: "3be9c8d3-a431-4c72-94ab-dd15bf815990") : secret "openshift-state-metrics-tls" not found Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782222 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782253 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782294 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be9c8d3-a431-4c72-94ab-dd15bf815990-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782335 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5fefb73d-befb-4eac-ba98-719b13989a8b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782374 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv6l8\" (UniqueName: \"kubernetes.io/projected/3be9c8d3-a431-4c72-94ab-dd15bf815990-kube-api-access-jv6l8\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782429 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54448918-e070-4b17-8bc3-010541c874b9-metrics-client-ca\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782458 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782491 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782523 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fefb73d-befb-4eac-ba98-719b13989a8b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.782595 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-wtmp\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.783149 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.782751 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-wtmp\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.783149 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:41.782832 2575 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 09 14:00:41.783149 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:41.782909 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-tls podName:5fefb73d-befb-4eac-ba98-719b13989a8b nodeName:}" failed. No retries permitted until 2026-07-09 14:00:42.28289408 +0000 UTC m=+306.136845555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-5z4jh" (UID: "5fefb73d-befb-4eac-ba98-719b13989a8b") : secret "kube-state-metrics-tls" not found Jul 09 14:00:41.783908 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.783466 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54448918-e070-4b17-8bc3-010541c874b9-sys\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.783908 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.783538 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.783908 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.783767 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-textfile\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.784183 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.784068 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54448918-e070-4b17-8bc3-010541c874b9-metrics-client-ca\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.784183 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.784116 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5fefb73d-befb-4eac-ba98-719b13989a8b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.784842 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.784811 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.785337 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.785316 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fefb73d-befb-4eac-ba98-719b13989a8b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.785476 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.785332 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3be9c8d3-a431-4c72-94ab-dd15bf815990-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.785659 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.785639 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-tls\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.787664 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.787639 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3be9c8d3-a431-4c72-94ab-dd15bf815990-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.787773 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.787739 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54448918-e070-4b17-8bc3-010541c874b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.788039 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.788019 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.796415 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.796347 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6vpg\" (UniqueName: \"kubernetes.io/projected/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-api-access-z6vpg\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:41.796714 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.796665 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv6l8\" (UniqueName: \"kubernetes.io/projected/3be9c8d3-a431-4c72-94ab-dd15bf815990-kube-api-access-jv6l8\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:41.797119 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.797097 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svj6d\" (UniqueName: \"kubernetes.io/projected/54448918-e070-4b17-8bc3-010541c874b9-kube-api-access-svj6d\") pod \"node-exporter-7bf4x\" (UID: \"54448918-e070-4b17-8bc3-010541c874b9\") " pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.911762 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.911715 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7bf4x" Jul 09 14:00:41.925227 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:41.925188 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54448918_e070_4b17_8bc3_010541c874b9.slice/crio-1cdf98d54e88d3961177ff21e54110e0010ae6518e0c596d503fd2ef36ac61a2 WatchSource:0}: Error finding container 1cdf98d54e88d3961177ff21e54110e0010ae6518e0c596d503fd2ef36ac61a2: Status 404 returned error can't find the container with id 1cdf98d54e88d3961177ff21e54110e0010ae6518e0c596d503fd2ef36ac61a2 Jul 09 14:00:41.927148 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:41.927128 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:00:42.288145 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.288107 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:42.288304 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.288193 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3be9c8d3-a431-4c72-94ab-dd15bf815990-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:42.290873 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.290821 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fefb73d-befb-4eac-ba98-719b13989a8b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5z4jh\" (UID: \"5fefb73d-befb-4eac-ba98-719b13989a8b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:42.290981 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.290882 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3be9c8d3-a431-4c72-94ab-dd15bf815990-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-jksvq\" (UID: \"3be9c8d3-a431-4c72-94ab-dd15bf815990\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:42.459572 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.459534 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" Jul 09 14:00:42.480349 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.477518 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" Jul 09 14:00:42.541211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.541079 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7bf4x" event={"ID":"54448918-e070-4b17-8bc3-010541c874b9","Type":"ContainerStarted","Data":"1cdf98d54e88d3961177ff21e54110e0010ae6518e0c596d503fd2ef36ac61a2"} Jul 09 14:00:42.666459 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.666419 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:00:42.671985 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.671954 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.676013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.675866 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 14:00:42.676013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.675933 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9wcsm\"" Jul 09 14:00:42.676546 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.676049 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 14:00:42.676546 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.676332 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 14:00:42.676546 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.676364 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 14:00:42.676546 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.676390 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 14:00:42.676546 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.676525 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 14:00:42.677053 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.676332 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 14:00:42.677053 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.676724 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 14:00:42.677053 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.676724 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 14:00:42.701926 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.700598 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh"] Jul 09 14:00:42.704839 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.704805 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:00:42.706232 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.706154 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq"] Jul 09 14:00:42.711564 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:42.711531 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be9c8d3_a431_4c72_94ab_dd15bf815990.slice/crio-b7b5ace33cda1ab6db95f72b40f3972006d391e272fa1da2f76a0ae32beb8a8f WatchSource:0}: Error finding container b7b5ace33cda1ab6db95f72b40f3972006d391e272fa1da2f76a0ae32beb8a8f: Status 404 returned error can't find the container with id b7b5ace33cda1ab6db95f72b40f3972006d391e272fa1da2f76a0ae32beb8a8f Jul 09 14:00:42.712239 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:42.712213 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fefb73d_befb_4eac_ba98_719b13989a8b.slice/crio-1c500f95026378897107b169c0e31e3ef093386e8ba98ae5c46c99dcd9c4284e WatchSource:0}: Error finding container 1c500f95026378897107b169c0e31e3ef093386e8ba98ae5c46c99dcd9c4284e: Status 404 returned error can't find the container with id 1c500f95026378897107b169c0e31e3ef093386e8ba98ae5c46c99dcd9c4284e Jul 09 14:00:42.793881 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.793698 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.793881 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.793772 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794088 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.793861 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794088 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794043 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794088 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794076 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794247 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794106 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794247 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794132 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794247 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794172 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-volume\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794247 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794200 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-web-config\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794247 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794226 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-out\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794552 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794259 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794552 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfrh2\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-kube-api-access-tfrh2\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.794552 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.794327 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.895743 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.895702 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.895976 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.895755 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfrh2\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-kube-api-access-tfrh2\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.895976 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.895798 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.895976 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.895882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.895976 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.895932 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.895976 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.895966 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.896242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.895995 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.896242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.896023 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.896242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.896050 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.896242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.896078 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.896242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.896113 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-volume\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.896242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.896141 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-web-config\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.896242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.896167 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-out\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.896583 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:42.896290 2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 14:00:42.896583 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:42.896366 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls podName:c1649ecf-f977-406b-b55d-c08f23c9f0af nodeName:}" failed. No retries permitted until 2026-07-09 14:00:43.396343667 +0000 UTC m=+307.250295150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af") : secret "alertmanager-main-tls" not found Jul 09 14:00:42.896697 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.896646 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.897379 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.897350 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.897987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.897937 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.901128 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.901101 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-out\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.902548 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.901657 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.902548 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.902122 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.902548 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.902496 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.902776 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.902608 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-web-config\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.902883 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.902841 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.904221 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.904189 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.904328 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.904225 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-volume\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:42.907523 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:42.907499 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfrh2\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-kube-api-access-tfrh2\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:43.400608 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:43.400505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:43.400797 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:43.400702 2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 14:00:43.400797 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:00:43.400790 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls podName:c1649ecf-f977-406b-b55d-c08f23c9f0af nodeName:}" failed. No retries permitted until 2026-07-09 14:00:44.400768436 +0000 UTC m=+308.254719904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af") : secret "alertmanager-main-tls" not found Jul 09 14:00:43.552259 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:43.552215 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" event={"ID":"3be9c8d3-a431-4c72-94ab-dd15bf815990","Type":"ContainerStarted","Data":"7b918f67e8190e1088335108d592be569cb237825a6d173ccee076f99352f8df"} Jul 09 14:00:43.552451 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:43.552332 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" event={"ID":"3be9c8d3-a431-4c72-94ab-dd15bf815990","Type":"ContainerStarted","Data":"7ed2587d0f34b5d52c08afe16133007b8859a7aff347370402ce3fd7aac1e680"} Jul 09 14:00:43.552451 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:43.552363 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" event={"ID":"3be9c8d3-a431-4c72-94ab-dd15bf815990","Type":"ContainerStarted","Data":"b7b5ace33cda1ab6db95f72b40f3972006d391e272fa1da2f76a0ae32beb8a8f"} Jul 09 14:00:43.554300 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:43.554264 2575 generic.go:356] "Generic (PLEG): container finished" podID="54448918-e070-4b17-8bc3-010541c874b9" containerID="1da80281d1f7a992fa178ee9371d6f4416db3a492a0769f015ac3a417b70fbb8" exitCode=0 Jul 09 14:00:43.554449 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:43.554359 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7bf4x" event={"ID":"54448918-e070-4b17-8bc3-010541c874b9","Type":"ContainerDied","Data":"1da80281d1f7a992fa178ee9371d6f4416db3a492a0769f015ac3a417b70fbb8"} Jul 09 14:00:43.556231 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:43.556203 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" event={"ID":"5fefb73d-befb-4eac-ba98-719b13989a8b","Type":"ContainerStarted","Data":"1c500f95026378897107b169c0e31e3ef093386e8ba98ae5c46c99dcd9c4284e"} Jul 09 14:00:43.752272 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:43.752169 2575 patch_prober.go:28] interesting pod/image-registry-958d69fff-j2glx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 14:00:43.752272 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:43.752238 2575 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-958d69fff-j2glx" podUID="0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:00:44.411738 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.411700 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:44.414239 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.414214 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:44.486342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.486301 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:00:44.563747 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.563707 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7bf4x" event={"ID":"54448918-e070-4b17-8bc3-010541c874b9","Type":"ContainerStarted","Data":"80061447d887576b536755cbfa3ef827bf457e6dfcfe52db5e1cb631b4d45be3"} Jul 09 14:00:44.563976 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.563770 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7bf4x" event={"ID":"54448918-e070-4b17-8bc3-010541c874b9","Type":"ContainerStarted","Data":"578c48a0217a9b15947bd095dcaa40c48b3b6e14fbb02e1b7338ba52b06635d1"} Jul 09 14:00:44.567361 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.567322 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" event={"ID":"5fefb73d-befb-4eac-ba98-719b13989a8b","Type":"ContainerStarted","Data":"5e7b50a9b1671cc470115b9b2eaf8d0ef2571a0e158817ee86ced9fd72be2d07"} Jul 09 14:00:44.567533 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.567405 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" event={"ID":"5fefb73d-befb-4eac-ba98-719b13989a8b","Type":"ContainerStarted","Data":"e2fc9b54e2222621b0c4baa9698fb35d2845d9b48ff0c64c475582d70a72c8f5"} Jul 09 14:00:44.567533 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.567439 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" event={"ID":"5fefb73d-befb-4eac-ba98-719b13989a8b","Type":"ContainerStarted","Data":"9f091734017ba532314bc8af969f16fb76245dce97df930ca6cf2d92377794ce"} Jul 09 14:00:44.590982 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.590000 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7bf4x" podStartSLOduration=2.377928274 podStartE2EDuration="3.58997444s" totalImagesPullingTime="1.129491457s" totalInitContainerRuntime="82.554709ms" isStatefulPod=true podCreationTimestamp="2026-07-09 14:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:44.587707233 +0000 UTC m=+308.441658711" watchObservedRunningTime="2026-07-09 14:00:44.58997444 +0000 UTC m=+308.443925917" Jul 09 14:00:44.613360 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.611968 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5z4jh" podStartSLOduration=2.30642271 podStartE2EDuration="3.611948909s" totalImagesPullingTime="1.305526199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:44.610634717 +0000 UTC m=+308.464586195" watchObservedRunningTime="2026-07-09 14:00:44.611948909 +0000 UTC m=+308.465900385" Jul 09 14:00:44.801151 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:44.800713 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:00:44.804045 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:44.803996 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1649ecf_f977_406b_b55d_c08f23c9f0af.slice/crio-99f6e2bd655d76671e1e3020a2012b22f38b6590adbf12fa7420a6fd02986b79 WatchSource:0}: Error finding container 99f6e2bd655d76671e1e3020a2012b22f38b6590adbf12fa7420a6fd02986b79: Status 404 returned error can't find the container with id 99f6e2bd655d76671e1e3020a2012b22f38b6590adbf12fa7420a6fd02986b79 Jul 09 14:00:45.574806 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:45.574767 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" event={"ID":"3be9c8d3-a431-4c72-94ab-dd15bf815990","Type":"ContainerStarted","Data":"a3d4a5c8192426157f971659ecbb87b6f8652e29fcadd7148e034393a6ef43d6"} Jul 09 14:00:45.576257 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:45.576229 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerStarted","Data":"99f6e2bd655d76671e1e3020a2012b22f38b6590adbf12fa7420a6fd02986b79"} Jul 09 14:00:45.592960 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:45.592901 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-jksvq" podStartSLOduration=2.974342052 podStartE2EDuration="4.592881743s" totalImagesPullingTime="1.618539691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:45.591569091 +0000 UTC m=+309.445520591" watchObservedRunningTime="2026-07-09 14:00:45.592881743 +0000 UTC m=+309.446833220" Jul 09 14:00:45.631965 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:45.631926 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 14:00:46.581762 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.581719 2575 generic.go:356] "Generic (PLEG): container finished" podID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerID="cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310" exitCode=0 Jul 09 14:00:46.582234 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.581796 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerDied","Data":"cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310"} Jul 09 14:00:46.788611 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.788570 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5d48595994-rkg6b"] Jul 09 14:00:46.792154 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.792128 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:46.795162 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.795126 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 14:00:46.795325 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.795160 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 14:00:46.795325 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.795171 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-f9dn6\"" Jul 09 14:00:46.795325 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.795160 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 14:00:46.795325 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.795126 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 14:00:46.795533 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.795415 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 14:00:46.801160 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.801138 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 14:00:46.809627 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.809597 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d48595994-rkg6b"] Jul 09 14:00:46.945643 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.945549 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-metrics-client-ca\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:46.945643 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.945593 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8vmr\" (UniqueName: \"kubernetes.io/projected/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-kube-api-access-f8vmr\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:46.945643 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.945622 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-telemeter-client-tls\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:46.945929 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.945680 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-serving-certs-ca-bundle\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:46.945929 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.945700 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-federate-client-tls\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:46.945929 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.945732 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:46.945929 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.945756 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-secret-telemeter-client\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:46.945929 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:46.945837 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.047209 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.047169 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-secret-telemeter-client\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.047406 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.047253 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.047406 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.047307 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-metrics-client-ca\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.047406 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.047336 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8vmr\" (UniqueName: \"kubernetes.io/projected/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-kube-api-access-f8vmr\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.047406 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.047373 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-telemeter-client-tls\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.047605 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.047413 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-serving-certs-ca-bundle\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.047605 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.047445 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-federate-client-tls\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.047605 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.047472 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.048302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.048255 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-serving-certs-ca-bundle\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.048434 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.048419 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-metrics-client-ca\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.048820 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.048786 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.050140 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.050117 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-telemeter-client-tls\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.050263 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.050228 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-federate-client-tls\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.050327 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.050288 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-secret-telemeter-client\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.050327 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.050299 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.062733 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.062702 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8vmr\" (UniqueName: \"kubernetes.io/projected/7cfeed82-6ea1-4547-b5d6-ed81cee08b22-kube-api-access-f8vmr\") pod \"telemeter-client-5d48595994-rkg6b\" (UID: \"7cfeed82-6ea1-4547-b5d6-ed81cee08b22\") " pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.102707 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.102667 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" Jul 09 14:00:47.275671 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.275572 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d48595994-rkg6b"] Jul 09 14:00:47.278933 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:00:47.278906 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cfeed82_6ea1_4547_b5d6_ed81cee08b22.slice/crio-436b08cd5723ee81b5539e2a471aa88986730b102a0d4d576b71ebb2e9b3783d WatchSource:0}: Error finding container 436b08cd5723ee81b5539e2a471aa88986730b102a0d4d576b71ebb2e9b3783d: Status 404 returned error can't find the container with id 436b08cd5723ee81b5539e2a471aa88986730b102a0d4d576b71ebb2e9b3783d Jul 09 14:00:47.472825 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.472439 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 14:00:47.586516 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:47.586482 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" event={"ID":"7cfeed82-6ea1-4547-b5d6-ed81cee08b22","Type":"ContainerStarted","Data":"436b08cd5723ee81b5539e2a471aa88986730b102a0d4d576b71ebb2e9b3783d"} Jul 09 14:00:49.599962 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:49.599885 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" event={"ID":"7cfeed82-6ea1-4547-b5d6-ed81cee08b22","Type":"ContainerStarted","Data":"2c774219b8002122df63f483abc3721aaf2095df27ebfaacdcdb807837502588"} Jul 09 14:00:49.603219 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:49.603123 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerStarted","Data":"254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78"} Jul 09 14:00:49.603219 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:49.603215 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerStarted","Data":"749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713"} Jul 09 14:00:49.603490 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:49.603263 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerStarted","Data":"b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5"} Jul 09 14:00:49.603490 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:49.603307 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerStarted","Data":"e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466"} Jul 09 14:00:50.610070 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:50.610029 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" event={"ID":"7cfeed82-6ea1-4547-b5d6-ed81cee08b22","Type":"ContainerStarted","Data":"e2eb63a579b04cdc41fe3de5baf3bed2da25b7b3593f098fb605899236507258"} Jul 09 14:00:50.610545 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:50.610169 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" event={"ID":"7cfeed82-6ea1-4547-b5d6-ed81cee08b22","Type":"ContainerStarted","Data":"b3c914f66bb18fee5984e75661b77bee1b88ace636be1c11c821e941d35d4de0"} Jul 09 14:00:50.613706 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:50.613676 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerStarted","Data":"9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5"} Jul 09 14:00:50.637773 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:50.637720 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5d48595994-rkg6b" podStartSLOduration=2.438912017 podStartE2EDuration="4.637705377s" totalImagesPullingTime="2.19879336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:50.637413151 +0000 UTC m=+314.491364638" watchObservedRunningTime="2026-07-09 14:00:50.637705377 +0000 UTC m=+314.491656864" Jul 09 14:00:51.621211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:51.621173 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerStarted","Data":"3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8"} Jul 09 14:00:51.656601 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:51.656545 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.123413003 podStartE2EDuration="9.656527616s" totalImagesPullingTime="4.513450336s" totalInitContainerRuntime="19.664277ms" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:51.654786777 +0000 UTC m=+315.508738249" watchObservedRunningTime="2026-07-09 14:00:51.656527616 +0000 UTC m=+315.510479119" Jul 09 14:00:52.526927 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:00:52.526887 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-958d69fff-j2glx"] Jul 09 14:01:06.348985 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.348948 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dcb79ff78-bhttp"] Jul 09 14:01:06.373814 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.373779 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dcb79ff78-bhttp"] Jul 09 14:01:06.373998 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.373908 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.377142 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.377116 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 14:01:06.378288 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.378262 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 14:01:06.378415 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.378351 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-p6gtz\"" Jul 09 14:01:06.378415 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.378351 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 14:01:06.378415 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.378404 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 14:01:06.378574 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.378357 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 14:01:06.378574 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.378503 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 14:01:06.378679 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.378667 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 14:01:06.415610 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.415571 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-config\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.415610 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.415613 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6l7x\" (UniqueName: \"kubernetes.io/projected/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-kube-api-access-l6l7x\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.415893 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.415635 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-oauth-config\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.415893 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.415698 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-service-ca\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.415893 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.415722 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-serving-cert\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.415893 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.415776 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-oauth-serving-cert\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.516281 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.516225 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-serving-cert\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.516281 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.516282 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-oauth-serving-cert\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.516532 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.516342 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-config\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.516532 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.516365 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6l7x\" (UniqueName: \"kubernetes.io/projected/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-kube-api-access-l6l7x\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.516532 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.516393 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-oauth-config\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.516532 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.516461 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-service-ca\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.517271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.517178 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-oauth-serving-cert\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.517271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.517228 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-service-ca\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.517271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.517254 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-config\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.518999 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.518975 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-oauth-config\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.519121 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.519038 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-serving-cert\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.526437 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.526413 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6l7x\" (UniqueName: \"kubernetes.io/projected/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-kube-api-access-l6l7x\") pod \"console-dcb79ff78-bhttp\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.684327 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.684238 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:06.822503 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:06.822471 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dcb79ff78-bhttp"] Jul 09 14:01:06.824784 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:01:06.824756 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dcebc14_520c_48fb_8a16_c5786f0d1d7d.slice/crio-593b25892c87ed9d2a282ce278d5e48a155e521173bdd89aefdfcebe4053be9a WatchSource:0}: Error finding container 593b25892c87ed9d2a282ce278d5e48a155e521173bdd89aefdfcebe4053be9a: Status 404 returned error can't find the container with id 593b25892c87ed9d2a282ce278d5e48a155e521173bdd89aefdfcebe4053be9a Jul 09 14:01:07.685218 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:07.685167 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dcb79ff78-bhttp" event={"ID":"2dcebc14-520c-48fb-8a16-c5786f0d1d7d","Type":"ContainerStarted","Data":"593b25892c87ed9d2a282ce278d5e48a155e521173bdd89aefdfcebe4053be9a"} Jul 09 14:01:10.700016 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:10.699934 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dcb79ff78-bhttp" event={"ID":"2dcebc14-520c-48fb-8a16-c5786f0d1d7d","Type":"ContainerStarted","Data":"a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47"} Jul 09 14:01:10.719380 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:10.719326 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dcb79ff78-bhttp" podStartSLOduration=1.698149023 podStartE2EDuration="4.719307775s" totalImagesPullingTime="3.021158752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:01:10.718001941 +0000 UTC m=+334.571953416" watchObservedRunningTime="2026-07-09 14:01:10.719307775 +0000 UTC m=+334.573259361" Jul 09 14:01:14.717839 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:14.717805 2575 generic.go:356] "Generic (PLEG): container finished" podID="178e8cc7-9a8b-4793-99da-e3af1543c23d" containerID="b736c40337672d85ef4ef7273d8550981d80fe614ff4dd5f07ec131ae72b771b" exitCode=0 Jul 09 14:01:14.718312 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:14.717889 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" event={"ID":"178e8cc7-9a8b-4793-99da-e3af1543c23d","Type":"ContainerDied","Data":"b736c40337672d85ef4ef7273d8550981d80fe614ff4dd5f07ec131ae72b771b"} Jul 09 14:01:14.718399 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:14.718310 2575 scope.go:117] "RemoveContainer" containerID="b736c40337672d85ef4ef7273d8550981d80fe614ff4dd5f07ec131ae72b771b" Jul 09 14:01:15.724113 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:15.724067 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-tvd65" event={"ID":"178e8cc7-9a8b-4793-99da-e3af1543c23d","Type":"ContainerStarted","Data":"c611776089cd04037bd07d0b0c6688fafc23a3e92cfebb7cdf4721c79045aee7"} Jul 09 14:01:16.685302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:16.685269 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:16.685496 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:16.685317 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:16.690284 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:16.690257 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:16.731743 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:16.731717 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:17.554305 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.554266 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-958d69fff-j2glx" podUID="0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" containerName="registry" containerID="cri-o://35546c15239addfb7e78cee7a64ab97f8287f9bd1a5eb6fe89fcb99e95f41b33" gracePeriod=30 Jul 09 14:01:17.734007 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.733975 2575 generic.go:356] "Generic (PLEG): container finished" podID="0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" containerID="35546c15239addfb7e78cee7a64ab97f8287f9bd1a5eb6fe89fcb99e95f41b33" exitCode=0 Jul 09 14:01:17.734384 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.734057 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-958d69fff-j2glx" event={"ID":"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2","Type":"ContainerDied","Data":"35546c15239addfb7e78cee7a64ab97f8287f9bd1a5eb6fe89fcb99e95f41b33"} Jul 09 14:01:17.816279 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.816223 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 14:01:17.911244 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.911205 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-image-registry-private-configuration\") pod \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " Jul 09 14:01:17.911442 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.911258 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dq2t7\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-kube-api-access-dq2t7\") pod \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " Jul 09 14:01:17.911442 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.911309 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-installation-pull-secrets\") pod \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " Jul 09 14:01:17.911442 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.911336 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-ca-trust-extracted\") pod \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " Jul 09 14:01:17.911442 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.911393 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-bound-sa-token\") pod \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " Jul 09 14:01:17.911442 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.911419 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-trusted-ca\") pod \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " Jul 09 14:01:17.911740 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.911444 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") pod \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " Jul 09 14:01:17.911740 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.911472 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-certificates\") pod \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\" (UID: \"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2\") " Jul 09 14:01:17.911943 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.911879 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:01:17.912234 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.912147 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:01:17.914445 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.914412 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:01:17.914559 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.914457 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:01:17.914559 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.914524 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-kube-api-access-dq2t7" (OuterVolumeSpecName: "kube-api-access-dq2t7") pod "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2"). InnerVolumeSpecName "kube-api-access-dq2t7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:01:17.914559 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.914535 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:01:17.914741 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.914653 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:01:17.920236 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:17.920212 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" (UID: "0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:01:18.013107 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.013068 2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-installation-pull-secrets\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:18.013107 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.013096 2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-ca-trust-extracted\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:18.013107 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.013109 2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-bound-sa-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:18.013107 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.013118 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-trusted-ca\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:18.013367 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.013126 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-tls\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:18.013367 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.013135 2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-registry-certificates\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:18.013367 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.013145 2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-image-registry-private-configuration\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:18.013367 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.013154 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dq2t7\" (UniqueName: \"kubernetes.io/projected/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2-kube-api-access-dq2t7\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:18.739998 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.739958 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-958d69fff-j2glx" event={"ID":"0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2","Type":"ContainerDied","Data":"7d3a1762eb978c86751493839dea9b8bf5e9d24a109ee0afd55215da6fed6fc3"} Jul 09 14:01:18.739998 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.739984 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-958d69fff-j2glx" Jul 09 14:01:18.740548 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.740031 2575 scope.go:117] "RemoveContainer" containerID="35546c15239addfb7e78cee7a64ab97f8287f9bd1a5eb6fe89fcb99e95f41b33" Jul 09 14:01:18.760715 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.760681 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-958d69fff-j2glx"] Jul 09 14:01:18.765248 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:18.765214 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-958d69fff-j2glx"] Jul 09 14:01:20.637573 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:20.637542 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" path="/var/lib/kubelet/pods/0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2/volumes" Jul 09 14:01:26.778268 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:26.778183 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dcb79ff78-bhttp"] Jul 09 14:01:50.610874 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:50.610794 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 14:01:50.613354 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:50.613330 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1ac675-9682-4468-993d-3d5183940a54-metrics-certs\") pod \"network-metrics-daemon-t2rdv\" (UID: \"4e1ac675-9682-4468-993d-3d5183940a54\") " pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 14:01:50.734928 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:50.734893 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qkthd\"" Jul 09 14:01:50.743154 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:50.743129 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t2rdv" Jul 09 14:01:50.884202 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:50.884176 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t2rdv"] Jul 09 14:01:50.886710 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:01:50.886682 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1ac675_9682_4468_993d_3d5183940a54.slice/crio-b8c202988d31fc9374082f4b5cc348bb2a8358f5f9855c8d27d33cc848f890c0 WatchSource:0}: Error finding container b8c202988d31fc9374082f4b5cc348bb2a8358f5f9855c8d27d33cc848f890c0: Status 404 returned error can't find the container with id b8c202988d31fc9374082f4b5cc348bb2a8358f5f9855c8d27d33cc848f890c0 Jul 09 14:01:51.803365 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:51.803322 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-dcb79ff78-bhttp" podUID="2dcebc14-520c-48fb-8a16-c5786f0d1d7d" containerName="console" containerID="cri-o://a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47" gracePeriod=15 Jul 09 14:01:51.878689 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:51.878643 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t2rdv" event={"ID":"4e1ac675-9682-4468-993d-3d5183940a54","Type":"ContainerStarted","Data":"b8c202988d31fc9374082f4b5cc348bb2a8358f5f9855c8d27d33cc848f890c0"} Jul 09 14:01:52.167689 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.167664 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:52.221765 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.221126 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-service-ca\") pod \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " Jul 09 14:01:52.221765 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.221180 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-serving-cert\") pod \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " Jul 09 14:01:52.221765 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.221224 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-oauth-config\") pod \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " Jul 09 14:01:52.221765 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.221264 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-config\") pod \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " Jul 09 14:01:52.221765 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.221289 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6l7x\" (UniqueName: \"kubernetes.io/projected/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-kube-api-access-l6l7x\") pod \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " Jul 09 14:01:52.221765 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.221337 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-oauth-serving-cert\") pod \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\" (UID: \"2dcebc14-520c-48fb-8a16-c5786f0d1d7d\") " Jul 09 14:01:52.222334 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.222128 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2dcebc14-520c-48fb-8a16-c5786f0d1d7d" (UID: "2dcebc14-520c-48fb-8a16-c5786f0d1d7d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:01:52.222388 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.222362 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-config" (OuterVolumeSpecName: "console-config") pod "2dcebc14-520c-48fb-8a16-c5786f0d1d7d" (UID: "2dcebc14-520c-48fb-8a16-c5786f0d1d7d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:01:52.222577 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.222542 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-service-ca" (OuterVolumeSpecName: "service-ca") pod "2dcebc14-520c-48fb-8a16-c5786f0d1d7d" (UID: "2dcebc14-520c-48fb-8a16-c5786f0d1d7d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:01:52.225073 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.225035 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-kube-api-access-l6l7x" (OuterVolumeSpecName: "kube-api-access-l6l7x") pod "2dcebc14-520c-48fb-8a16-c5786f0d1d7d" (UID: "2dcebc14-520c-48fb-8a16-c5786f0d1d7d"). InnerVolumeSpecName "kube-api-access-l6l7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:01:52.225181 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.225082 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2dcebc14-520c-48fb-8a16-c5786f0d1d7d" (UID: "2dcebc14-520c-48fb-8a16-c5786f0d1d7d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:01:52.225181 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.225144 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2dcebc14-520c-48fb-8a16-c5786f0d1d7d" (UID: "2dcebc14-520c-48fb-8a16-c5786f0d1d7d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:01:52.322276 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.322239 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-service-ca\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:52.322276 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.322274 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-serving-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:52.322474 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.322287 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-oauth-config\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:52.322474 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.322295 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-console-config\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:52.322474 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.322305 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6l7x\" (UniqueName: \"kubernetes.io/projected/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-kube-api-access-l6l7x\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:52.322474 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.322318 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dcebc14-520c-48fb-8a16-c5786f0d1d7d-oauth-serving-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:01:52.884961 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.884912 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t2rdv" event={"ID":"4e1ac675-9682-4468-993d-3d5183940a54","Type":"ContainerStarted","Data":"0292bc8958b3d7e8751a999564848b0406ac24cca3fdefb3b3fad8af39ec26aa"} Jul 09 14:01:52.885405 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.884978 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t2rdv" event={"ID":"4e1ac675-9682-4468-993d-3d5183940a54","Type":"ContainerStarted","Data":"8509506bba6f5ab61d5d459b1aa6bbe21b6c16118b867ad4717c43ebe8c1a928"} Jul 09 14:01:52.886297 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.886276 2575 generic.go:356] "Generic (PLEG): container finished" podID="2dcebc14-520c-48fb-8a16-c5786f0d1d7d" containerID="a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47" exitCode=0 Jul 09 14:01:52.886439 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.886329 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dcb79ff78-bhttp" Jul 09 14:01:52.886439 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.886350 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dcb79ff78-bhttp" event={"ID":"2dcebc14-520c-48fb-8a16-c5786f0d1d7d","Type":"ContainerDied","Data":"a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47"} Jul 09 14:01:52.886439 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.886380 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dcb79ff78-bhttp" event={"ID":"2dcebc14-520c-48fb-8a16-c5786f0d1d7d","Type":"ContainerDied","Data":"593b25892c87ed9d2a282ce278d5e48a155e521173bdd89aefdfcebe4053be9a"} Jul 09 14:01:52.886439 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.886399 2575 scope.go:117] "RemoveContainer" containerID="a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47" Jul 09 14:01:52.894834 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.894814 2575 scope.go:117] "RemoveContainer" containerID="a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47" Jul 09 14:01:52.895142 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:01:52.895120 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47\": container with ID starting with a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47 not found: ID does not exist" containerID="a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47" Jul 09 14:01:52.895191 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.895150 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47"} err="failed to get container status \"a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47\": rpc error: code = NotFound desc = could not find container \"a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47\": container with ID starting with a5144e5089206102f5305829ea6c8a7256ab01d90295f18d746000f776399f47 not found: ID does not exist" Jul 09 14:01:52.906943 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.906894 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t2rdv" podStartSLOduration=374.603897038 podStartE2EDuration="6m15.906880219s" totalImagesPullingTime="1.302983181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:55:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:01:52.906462299 +0000 UTC m=+376.760413777" watchObservedRunningTime="2026-07-09 14:01:52.906880219 +0000 UTC m=+376.760831695" Jul 09 14:01:52.925527 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.925488 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dcb79ff78-bhttp"] Jul 09 14:01:52.933020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:52.932996 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dcb79ff78-bhttp"] Jul 09 14:01:54.636795 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:01:54.636760 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dcebc14-520c-48fb-8a16-c5786f0d1d7d" path="/var/lib/kubelet/pods/2dcebc14-520c-48fb-8a16-c5786f0d1d7d/volumes" Jul 09 14:02:02.179062 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.179027 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:02:02.179476 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.179373 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="alertmanager" containerID="cri-o://e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466" gracePeriod=120 Jul 09 14:02:02.179476 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.179447 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy-metric" containerID="cri-o://9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5" gracePeriod=120 Jul 09 14:02:02.179585 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.179481 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy-web" containerID="cri-o://749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713" gracePeriod=120 Jul 09 14:02:02.179585 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.179509 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="prom-label-proxy" containerID="cri-o://3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8" gracePeriod=120 Jul 09 14:02:02.179585 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.179543 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="config-reloader" containerID="cri-o://b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5" gracePeriod=120 Jul 09 14:02:02.179734 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.179490 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy" containerID="cri-o://254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78" gracePeriod=120 Jul 09 14:02:02.932529 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.932497 2575 generic.go:356] "Generic (PLEG): container finished" podID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerID="3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8" exitCode=0 Jul 09 14:02:02.932529 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.932524 2575 generic.go:356] "Generic (PLEG): container finished" podID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerID="254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78" exitCode=0 Jul 09 14:02:02.932529 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.932530 2575 generic.go:356] "Generic (PLEG): container finished" podID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerID="b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5" exitCode=0 Jul 09 14:02:02.932529 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.932536 2575 generic.go:356] "Generic (PLEG): container finished" podID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerID="e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466" exitCode=0 Jul 09 14:02:02.932796 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.932558 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerDied","Data":"3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8"} Jul 09 14:02:02.932796 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.932607 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerDied","Data":"254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78"} Jul 09 14:02:02.932796 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.932632 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerDied","Data":"b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5"} Jul 09 14:02:02.932796 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:02.932657 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerDied","Data":"e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466"} Jul 09 14:02:03.426086 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.426054 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:03.499452 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499420 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-out\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499630 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499469 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfrh2\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-kube-api-access-tfrh2\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499630 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499487 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-metrics-client-ca\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499630 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499506 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-web-config\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499630 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499533 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-trusted-ca-bundle\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499630 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499596 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-web\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499945 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499635 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-cluster-tls-config\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499945 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499667 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-tls-assets\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499945 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499698 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499945 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499722 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499945 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499750 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499945 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499775 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-volume\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499945 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499809 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-main-db\") pod \"c1649ecf-f977-406b-b55d-c08f23c9f0af\" (UID: \"c1649ecf-f977-406b-b55d-c08f23c9f0af\") " Jul 09 14:02:03.499945 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.499926 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:02:03.500504 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.500190 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:02:03.500504 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.500217 2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-metrics-client-ca\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.500504 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.500441 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:02:03.503531 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.503401 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:02:03.503747 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.503655 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-kube-api-access-tfrh2" (OuterVolumeSpecName: "kube-api-access-tfrh2") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "kube-api-access-tfrh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:02:03.503883 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.503807 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-out" (OuterVolumeSpecName: "config-out") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:02:03.504119 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.504088 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:02:03.504119 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.504105 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:02:03.504826 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.504800 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:02:03.504942 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.504870 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:02:03.505172 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.505148 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-volume" (OuterVolumeSpecName: "config-volume") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:02:03.508281 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.508223 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:02:03.516101 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.516075 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-web-config" (OuterVolumeSpecName: "web-config") pod "c1649ecf-f977-406b-b55d-c08f23c9f0af" (UID: "c1649ecf-f977-406b-b55d-c08f23c9f0af"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:02:03.600669 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600567 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.600669 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600612 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-main-tls\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.600669 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600624 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.600669 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600635 2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-volume\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.600669 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600643 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-main-db\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.600669 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600651 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c1649ecf-f977-406b-b55d-c08f23c9f0af-config-out\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.600669 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600660 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfrh2\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-kube-api-access-tfrh2\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.601022 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600691 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-web-config\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.601022 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600699 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1649ecf-f977-406b-b55d-c08f23c9f0af-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.601022 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600708 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.601022 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600717 2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c1649ecf-f977-406b-b55d-c08f23c9f0af-cluster-tls-config\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.601022 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.600725 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c1649ecf-f977-406b-b55d-c08f23c9f0af-tls-assets\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:02:03.939338 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.939247 2575 generic.go:356] "Generic (PLEG): container finished" podID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerID="9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5" exitCode=0 Jul 09 14:02:03.939338 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.939274 2575 generic.go:356] "Generic (PLEG): container finished" podID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerID="749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713" exitCode=0 Jul 09 14:02:03.939338 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.939287 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerDied","Data":"9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5"} Jul 09 14:02:03.939610 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.939344 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:03.939610 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.939362 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerDied","Data":"749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713"} Jul 09 14:02:03.939610 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.939391 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c1649ecf-f977-406b-b55d-c08f23c9f0af","Type":"ContainerDied","Data":"99f6e2bd655d76671e1e3020a2012b22f38b6590adbf12fa7420a6fd02986b79"} Jul 09 14:02:03.939610 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.939419 2575 scope.go:117] "RemoveContainer" containerID="3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8" Jul 09 14:02:03.948365 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.948343 2575 scope.go:117] "RemoveContainer" containerID="9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5" Jul 09 14:02:03.955629 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.955610 2575 scope.go:117] "RemoveContainer" containerID="254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78" Jul 09 14:02:03.963363 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.963125 2575 scope.go:117] "RemoveContainer" containerID="749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713" Jul 09 14:02:03.963619 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.963600 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:02:03.968577 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.968553 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:02:03.971096 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.971080 2575 scope.go:117] "RemoveContainer" containerID="b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5" Jul 09 14:02:03.978410 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.978393 2575 scope.go:117] "RemoveContainer" containerID="e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466" Jul 09 14:02:03.985921 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.985901 2575 scope.go:117] "RemoveContainer" containerID="cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310" Jul 09 14:02:03.996803 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.996784 2575 scope.go:117] "RemoveContainer" containerID="3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8" Jul 09 14:02:03.997185 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:02:03.997165 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8\": container with ID starting with 3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8 not found: ID does not exist" containerID="3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8" Jul 09 14:02:03.997237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.997196 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8"} err="failed to get container status \"3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8\": rpc error: code = NotFound desc = could not find container \"3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8\": container with ID starting with 3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8 not found: ID does not exist" Jul 09 14:02:03.997237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.997217 2575 scope.go:117] "RemoveContainer" containerID="9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5" Jul 09 14:02:03.997500 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:02:03.997483 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5\": container with ID starting with 9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5 not found: ID does not exist" containerID="9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5" Jul 09 14:02:03.997547 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.997505 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5"} err="failed to get container status \"9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5\": rpc error: code = NotFound desc = could not find container \"9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5\": container with ID starting with 9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5 not found: ID does not exist" Jul 09 14:02:03.997547 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.997519 2575 scope.go:117] "RemoveContainer" containerID="254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78" Jul 09 14:02:03.997733 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:02:03.997713 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78\": container with ID starting with 254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78 not found: ID does not exist" containerID="254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78" Jul 09 14:02:03.997773 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.997737 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78"} err="failed to get container status \"254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78\": rpc error: code = NotFound desc = could not find container \"254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78\": container with ID starting with 254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78 not found: ID does not exist" Jul 09 14:02:03.997773 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.997749 2575 scope.go:117] "RemoveContainer" containerID="749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713" Jul 09 14:02:03.997975 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:02:03.997959 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713\": container with ID starting with 749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713 not found: ID does not exist" containerID="749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713" Jul 09 14:02:03.998025 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.997980 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713"} err="failed to get container status \"749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713\": rpc error: code = NotFound desc = could not find container \"749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713\": container with ID starting with 749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713 not found: ID does not exist" Jul 09 14:02:03.998025 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.997992 2575 scope.go:117] "RemoveContainer" containerID="b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5" Jul 09 14:02:03.998215 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:02:03.998202 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5\": container with ID starting with b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5 not found: ID does not exist" containerID="b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5" Jul 09 14:02:03.998251 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.998218 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5"} err="failed to get container status \"b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5\": rpc error: code = NotFound desc = could not find container \"b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5\": container with ID starting with b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5 not found: ID does not exist" Jul 09 14:02:03.998251 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.998229 2575 scope.go:117] "RemoveContainer" containerID="e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466" Jul 09 14:02:03.998466 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:02:03.998449 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466\": container with ID starting with e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466 not found: ID does not exist" containerID="e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466" Jul 09 14:02:03.998561 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.998470 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466"} err="failed to get container status \"e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466\": rpc error: code = NotFound desc = could not find container \"e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466\": container with ID starting with e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466 not found: ID does not exist" Jul 09 14:02:03.998561 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.998494 2575 scope.go:117] "RemoveContainer" containerID="cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310" Jul 09 14:02:03.998808 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:02:03.998784 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310\": container with ID starting with cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310 not found: ID does not exist" containerID="cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310" Jul 09 14:02:03.998905 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.998817 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310"} err="failed to get container status \"cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310\": rpc error: code = NotFound desc = could not find container \"cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310\": container with ID starting with cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310 not found: ID does not exist" Jul 09 14:02:03.998905 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.998834 2575 scope.go:117] "RemoveContainer" containerID="3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8" Jul 09 14:02:03.999332 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.999302 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8"} err="failed to get container status \"3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8\": rpc error: code = NotFound desc = could not find container \"3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8\": container with ID starting with 3a3fcf65517ca6f1812b46d828fc6c78d9b215a36fc97611860b52ace051bbe8 not found: ID does not exist" Jul 09 14:02:03.999332 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.999333 2575 scope.go:117] "RemoveContainer" containerID="9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5" Jul 09 14:02:03.999680 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.999655 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5"} err="failed to get container status \"9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5\": rpc error: code = NotFound desc = could not find container \"9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5\": container with ID starting with 9fc190529dbfbfc119699ee914466d406feb37c6dfd5f69b9098260dc0ae11e5 not found: ID does not exist" Jul 09 14:02:03.999772 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:03.999682 2575 scope.go:117] "RemoveContainer" containerID="254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78" Jul 09 14:02:04.000034 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.000014 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78"} err="failed to get container status \"254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78\": rpc error: code = NotFound desc = could not find container \"254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78\": container with ID starting with 254b2796ded317442c6b5fc59d06c64fb56cbb55aab938584dcbd02cc86b4e78 not found: ID does not exist" Jul 09 14:02:04.000125 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.000035 2575 scope.go:117] "RemoveContainer" containerID="749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713" Jul 09 14:02:04.000387 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.000327 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713"} err="failed to get container status \"749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713\": rpc error: code = NotFound desc = could not find container \"749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713\": container with ID starting with 749a32ca7257a8537359a52ba4431ada6e790593ef9a16a763bf000970b5d713 not found: ID does not exist" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.000951 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001352 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy-web" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001369 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy-web" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001383 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy-metric" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001391 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy-metric" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001410 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="prom-label-proxy" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001420 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="prom-label-proxy" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001430 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" containerName="registry" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001437 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" containerName="registry" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001446 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="alertmanager" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001453 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="alertmanager" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001462 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001469 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001478 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="config-reloader" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001485 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="config-reloader" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001495 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dcebc14-520c-48fb-8a16-c5786f0d1d7d" containerName="console" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001502 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dcebc14-520c-48fb-8a16-c5786f0d1d7d" containerName="console" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001512 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="init-config-reloader" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001520 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="init-config-reloader" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001585 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001597 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy-web" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001605 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="kube-rbac-proxy-metric" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001614 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="prom-label-proxy" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001624 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="alertmanager" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001631 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dcebc14-520c-48fb-8a16-c5786f0d1d7d" containerName="console" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001638 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c07a4c8-f329-4a1e-a5e0-36b0d4461cd2" containerName="registry" Jul 09 14:02:04.003271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.001645 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" containerName="config-reloader" Jul 09 14:02:04.007654 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.007633 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.011064 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.011041 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9wcsm\"" Jul 09 14:02:04.011513 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.011387 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 14:02:04.011748 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.011732 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 14:02:04.012054 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.012039 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 14:02:04.012356 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.012341 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 14:02:04.012667 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.012653 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 14:02:04.013012 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.012998 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 14:02:04.013107 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.013085 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 14:02:04.013303 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.013290 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 14:02:04.026655 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.018902 2575 scope.go:117] "RemoveContainer" containerID="b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5" Jul 09 14:02:04.026655 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.023150 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:02:04.026655 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.026340 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5"} err="failed to get container status \"b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5\": rpc error: code = NotFound desc = could not find container \"b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5\": container with ID starting with b2a37109aca807520c901054dd2c42db6536f1c760222e1328bea7ceddf559e5 not found: ID does not exist" Jul 09 14:02:04.026655 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.026371 2575 scope.go:117] "RemoveContainer" containerID="e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466" Jul 09 14:02:04.030034 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.027952 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466"} err="failed to get container status \"e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466\": rpc error: code = NotFound desc = could not find container \"e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466\": container with ID starting with e021890c81db39ade2b804bfe4d12157f5a2a249870e1919669ae324ae30b466 not found: ID does not exist" Jul 09 14:02:04.030034 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.027984 2575 scope.go:117] "RemoveContainer" containerID="cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310" Jul 09 14:02:04.030034 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.028299 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310"} err="failed to get container status \"cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310\": rpc error: code = NotFound desc = could not find container \"cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310\": container with ID starting with cc60d4a5777c5ab27036446e8e844dd41f33bc1550063fc356059c3b2793c310 not found: ID does not exist" Jul 09 14:02:04.030034 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.029997 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 14:02:04.105358 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105317 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105358 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105361 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29f91ba-3556-4603-bcee-4ba7a5e687a7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105574 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mftx6\" (UniqueName: \"kubernetes.io/projected/d29f91ba-3556-4603-bcee-4ba7a5e687a7-kube-api-access-mftx6\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105574 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105480 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105574 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105515 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d29f91ba-3556-4603-bcee-4ba7a5e687a7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105574 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105538 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105699 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105599 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105699 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105639 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d29f91ba-3556-4603-bcee-4ba7a5e687a7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105699 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105657 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105699 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105685 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d29f91ba-3556-4603-bcee-4ba7a5e687a7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105904 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105710 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-web-config\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105904 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105747 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d29f91ba-3556-4603-bcee-4ba7a5e687a7-config-out\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.105904 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.105763 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.206957 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.206822 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d29f91ba-3556-4603-bcee-4ba7a5e687a7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.206957 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.206913 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-web-config\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.206957 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.206953 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d29f91ba-3556-4603-bcee-4ba7a5e687a7-config-out\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.206976 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207008 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207038 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29f91ba-3556-4603-bcee-4ba7a5e687a7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207064 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mftx6\" (UniqueName: \"kubernetes.io/projected/d29f91ba-3556-4603-bcee-4ba7a5e687a7-kube-api-access-mftx6\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207095 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207125 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d29f91ba-3556-4603-bcee-4ba7a5e687a7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207156 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207237 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207197 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207547 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207243 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d29f91ba-3556-4603-bcee-4ba7a5e687a7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.207547 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207266 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.208252 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.207686 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d29f91ba-3556-4603-bcee-4ba7a5e687a7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.208252 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.208116 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d29f91ba-3556-4603-bcee-4ba7a5e687a7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.208252 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.208171 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d29f91ba-3556-4603-bcee-4ba7a5e687a7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.210226 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.210195 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d29f91ba-3556-4603-bcee-4ba7a5e687a7-config-out\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.210555 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.210532 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.210778 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.210752 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.210907 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.210808 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.210907 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.210834 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.211024 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.210908 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.211281 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.211262 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d29f91ba-3556-4603-bcee-4ba7a5e687a7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.211475 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.211458 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-web-config\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.211941 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.211924 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d29f91ba-3556-4603-bcee-4ba7a5e687a7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.224334 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.224299 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mftx6\" (UniqueName: \"kubernetes.io/projected/d29f91ba-3556-4603-bcee-4ba7a5e687a7-kube-api-access-mftx6\") pod \"alertmanager-main-0\" (UID: \"d29f91ba-3556-4603-bcee-4ba7a5e687a7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.329615 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.329561 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 14:02:04.481285 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.481252 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 14:02:04.483747 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:02:04.483711 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29f91ba_3556_4603_bcee_4ba7a5e687a7.slice/crio-49165b87e649222ebb6144d9f89be2753d952aa065ce6b398bb4ffd2e4177fda WatchSource:0}: Error finding container 49165b87e649222ebb6144d9f89be2753d952aa065ce6b398bb4ffd2e4177fda: Status 404 returned error can't find the container with id 49165b87e649222ebb6144d9f89be2753d952aa065ce6b398bb4ffd2e4177fda Jul 09 14:02:04.637196 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.637160 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1649ecf-f977-406b-b55d-c08f23c9f0af" path="/var/lib/kubelet/pods/c1649ecf-f977-406b-b55d-c08f23c9f0af/volumes" Jul 09 14:02:04.893338 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.893300 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7677c9696b-8fwdk"] Jul 09 14:02:04.896729 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.896707 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:04.899637 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.899617 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 14:02:04.899745 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.899638 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-p6gtz\"" Jul 09 14:02:04.899968 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.899946 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 14:02:04.900243 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.900221 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 14:02:04.900344 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.900281 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 14:02:04.900344 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.900285 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 14:02:04.900514 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.900341 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 14:02:04.900714 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.900697 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 14:02:04.904615 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.904593 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 14:02:04.913715 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.911402 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7677c9696b-8fwdk"] Jul 09 14:02:04.915780 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.915757 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-oauth-config\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:04.915917 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.915797 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-service-ca\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:04.915917 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.915888 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-trusted-ca-bundle\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:04.915917 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.915912 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-serving-cert\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:04.916055 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.915940 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-console-config\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:04.916104 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.916077 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47px5\" (UniqueName: \"kubernetes.io/projected/b950e6be-8973-4183-bd1a-1359bbb7597f-kube-api-access-47px5\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:04.916155 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.916129 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-oauth-serving-cert\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:04.946219 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.946178 2575 generic.go:356] "Generic (PLEG): container finished" podID="d29f91ba-3556-4603-bcee-4ba7a5e687a7" containerID="3d325429769174d68876709c796240db809e2d8590fec3c77666d8d20cdcea88" exitCode=0 Jul 09 14:02:04.946420 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.946245 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d29f91ba-3556-4603-bcee-4ba7a5e687a7","Type":"ContainerDied","Data":"3d325429769174d68876709c796240db809e2d8590fec3c77666d8d20cdcea88"} Jul 09 14:02:04.946420 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:04.946294 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d29f91ba-3556-4603-bcee-4ba7a5e687a7","Type":"ContainerStarted","Data":"49165b87e649222ebb6144d9f89be2753d952aa065ce6b398bb4ffd2e4177fda"} Jul 09 14:02:05.016978 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.016942 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47px5\" (UniqueName: \"kubernetes.io/projected/b950e6be-8973-4183-bd1a-1359bbb7597f-kube-api-access-47px5\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.017138 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.017007 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-oauth-serving-cert\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.017138 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.017067 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-oauth-config\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.017138 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.017125 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-service-ca\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.017307 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.017180 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-trusted-ca-bundle\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.017307 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.017210 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-serving-cert\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.017411 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.017319 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-console-config\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.018427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.018057 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-console-config\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.018427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.018272 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-service-ca\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.019002 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.018975 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-oauth-serving-cert\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.019093 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.019049 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-trusted-ca-bundle\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.019972 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.019948 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-serving-cert\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.020194 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.020147 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-oauth-config\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.025833 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.025794 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47px5\" (UniqueName: \"kubernetes.io/projected/b950e6be-8973-4183-bd1a-1359bbb7597f-kube-api-access-47px5\") pod \"console-7677c9696b-8fwdk\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.207012 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.206983 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:05.359541 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.359453 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7677c9696b-8fwdk"] Jul 09 14:02:05.362156 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:02:05.362125 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb950e6be_8973_4183_bd1a_1359bbb7597f.slice/crio-48d21d07a4c8ef0cd836e8954058983f2ab31d3aee411a7cbd749dc9f994d26e WatchSource:0}: Error finding container 48d21d07a4c8ef0cd836e8954058983f2ab31d3aee411a7cbd749dc9f994d26e: Status 404 returned error can't find the container with id 48d21d07a4c8ef0cd836e8954058983f2ab31d3aee411a7cbd749dc9f994d26e Jul 09 14:02:05.953959 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.953917 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d29f91ba-3556-4603-bcee-4ba7a5e687a7","Type":"ContainerStarted","Data":"0d648aaa4a581c094ca392a1c4d7d4015325bb3777eed4c82c87ed44d828dbb8"} Jul 09 14:02:05.954404 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.953997 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d29f91ba-3556-4603-bcee-4ba7a5e687a7","Type":"ContainerStarted","Data":"e75737d43660b66d880eac71065ed851aee947001ad58bb7908a8badd18aa662"} Jul 09 14:02:05.954404 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.954036 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d29f91ba-3556-4603-bcee-4ba7a5e687a7","Type":"ContainerStarted","Data":"644ab80694e535b670473588ba8088bb147d254bd302d9a7bb000209cff8f54b"} Jul 09 14:02:05.954404 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.954071 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d29f91ba-3556-4603-bcee-4ba7a5e687a7","Type":"ContainerStarted","Data":"60e912c2ff9fd0f78125c9735a79b99d9abfb42cdbdc95cb203f2109bcaf5e8c"} Jul 09 14:02:05.954404 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.954107 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d29f91ba-3556-4603-bcee-4ba7a5e687a7","Type":"ContainerStarted","Data":"efa6cef95dcee71e53d629963487dbd90a3b45d5c39178a4a0ad090bb7bd0a1f"} Jul 09 14:02:05.954404 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.954146 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d29f91ba-3556-4603-bcee-4ba7a5e687a7","Type":"ContainerStarted","Data":"4b50872ee23543e11f3775f6b465f9c8abf53b1adbceec91934cc27e5780fe8a"} Jul 09 14:02:05.955458 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.955435 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677c9696b-8fwdk" event={"ID":"b950e6be-8973-4183-bd1a-1359bbb7597f","Type":"ContainerStarted","Data":"de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada"} Jul 09 14:02:05.955548 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.955501 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677c9696b-8fwdk" event={"ID":"b950e6be-8973-4183-bd1a-1359bbb7597f","Type":"ContainerStarted","Data":"48d21d07a4c8ef0cd836e8954058983f2ab31d3aee411a7cbd749dc9f994d26e"} Jul 09 14:02:05.985661 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:05.985609 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.966423865 podStartE2EDuration="2.985592416s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.168551ms" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:05.984565157 +0000 UTC m=+389.838516636" watchObservedRunningTime="2026-07-09 14:02:05.985592416 +0000 UTC m=+389.839543903" Jul 09 14:02:06.006164 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:06.006103 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7677c9696b-8fwdk" podStartSLOduration=2.006079156 podStartE2EDuration="2.006079156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:02:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:02:06.003886809 +0000 UTC m=+389.857838285" watchObservedRunningTime="2026-07-09 14:02:06.006079156 +0000 UTC m=+389.860030651" Jul 09 14:02:15.207145 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:15.207097 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:15.207548 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:15.207174 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:15.212256 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:15.212226 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:02:16.000801 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:02:16.000774 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:03:15.791682 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:15.791645 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx"] Jul 09 14:03:15.795319 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:15.795302 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:15.798520 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:15.798493 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 14:03:15.799444 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:15.799426 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 14:03:15.799500 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:15.799451 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gmwc2\"" Jul 09 14:03:15.814428 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:15.814400 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx"] Jul 09 14:03:15.918690 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:15.918654 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:15.918690 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:15.918697 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:15.918956 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:15.918752 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dssm\" (UniqueName: \"kubernetes.io/projected/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-kube-api-access-6dssm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:16.019628 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:16.019587 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:16.019823 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:16.019648 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dssm\" (UniqueName: \"kubernetes.io/projected/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-kube-api-access-6dssm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:16.019823 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:16.019714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:16.020162 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:16.020140 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:16.020228 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:16.020169 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:16.030493 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:16.030465 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dssm\" (UniqueName: \"kubernetes.io/projected/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-kube-api-access-6dssm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:16.104547 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:16.104460 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:16.251153 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:16.251119 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx"] Jul 09 14:03:16.253783 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:03:16.253753 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd5ce086_5b5e_445e_8c6f_9ac94bfdd81c.slice/crio-2c9cc497b50a1b6780dd9e7e8215736a64ab734e079d2a33390270e2a45a8245 WatchSource:0}: Error finding container 2c9cc497b50a1b6780dd9e7e8215736a64ab734e079d2a33390270e2a45a8245: Status 404 returned error can't find the container with id 2c9cc497b50a1b6780dd9e7e8215736a64ab734e079d2a33390270e2a45a8245 Jul 09 14:03:17.235826 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:17.235785 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" event={"ID":"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c","Type":"ContainerStarted","Data":"2c9cc497b50a1b6780dd9e7e8215736a64ab734e079d2a33390270e2a45a8245"} Jul 09 14:03:25.273213 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:25.273180 2575 generic.go:356] "Generic (PLEG): container finished" podID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerID="57b6214c8375aa4bd7825c753c9060e1615c417799842070b597358c6c8957ac" exitCode=0 Jul 09 14:03:25.273785 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:25.273236 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" event={"ID":"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c","Type":"ContainerDied","Data":"57b6214c8375aa4bd7825c753c9060e1615c417799842070b597358c6c8957ac"} Jul 09 14:03:28.289523 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:28.289482 2575 generic.go:356] "Generic (PLEG): container finished" podID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerID="a7f0c2ed7535fb752fc0c7849b95ff6ea1b9f0fac50b6f831fd0e846832282ed" exitCode=0 Jul 09 14:03:28.289948 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:28.289568 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" event={"ID":"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c","Type":"ContainerDied","Data":"a7f0c2ed7535fb752fc0c7849b95ff6ea1b9f0fac50b6f831fd0e846832282ed"} Jul 09 14:03:39.339892 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:39.339830 2575 generic.go:356] "Generic (PLEG): container finished" podID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerID="ce1e317642a27fca4f5571e552b54aaa30d9b1337dea4c92e122151952a701bf" exitCode=0 Jul 09 14:03:39.340304 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:39.339940 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" event={"ID":"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c","Type":"ContainerDied","Data":"ce1e317642a27fca4f5571e552b54aaa30d9b1337dea4c92e122151952a701bf"} Jul 09 14:03:40.495641 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.495610 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:40.635010 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.634923 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-bundle\") pod \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " Jul 09 14:03:40.635010 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.634979 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dssm\" (UniqueName: \"kubernetes.io/projected/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-kube-api-access-6dssm\") pod \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " Jul 09 14:03:40.635267 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.635029 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-util\") pod \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\" (UID: \"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c\") " Jul 09 14:03:40.635646 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.635622 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-bundle" (OuterVolumeSpecName: "bundle") pod "cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" (UID: "cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:03:40.637361 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.637338 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-kube-api-access-6dssm" (OuterVolumeSpecName: "kube-api-access-6dssm") pod "cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" (UID: "cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c"). InnerVolumeSpecName "kube-api-access-6dssm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:03:40.640683 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.640656 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-util" (OuterVolumeSpecName: "util") pod "cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" (UID: "cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:03:40.736313 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.736279 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dssm\" (UniqueName: \"kubernetes.io/projected/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-kube-api-access-6dssm\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:03:40.736313 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.736309 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-util\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:03:40.736313 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:40.736319 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c-bundle\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:03:41.349187 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:41.349145 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" event={"ID":"cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c","Type":"ContainerDied","Data":"2c9cc497b50a1b6780dd9e7e8215736a64ab734e079d2a33390270e2a45a8245"} Jul 09 14:03:41.349354 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:41.349196 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408vjjx" Jul 09 14:03:41.349354 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:41.349202 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c9cc497b50a1b6780dd9e7e8215736a64ab734e079d2a33390270e2a45a8245" Jul 09 14:03:49.689415 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.689378 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59"] Jul 09 14:03:49.689906 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.689671 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerName="pull" Jul 09 14:03:49.689906 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.689682 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerName="pull" Jul 09 14:03:49.689906 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.689695 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerName="util" Jul 09 14:03:49.689906 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.689700 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerName="util" Jul 09 14:03:49.689906 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.689711 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerName="extract" Jul 09 14:03:49.689906 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.689716 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerName="extract" Jul 09 14:03:49.689906 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.689769 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd5ce086-5b5e-445e-8c6f-9ac94bfdd81c" containerName="extract" Jul 09 14:03:49.691723 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.691707 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" Jul 09 14:03:49.696703 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.696681 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 09 14:03:49.698181 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.698013 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:03:49.698489 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.698470 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-wcvcn\"" Jul 09 14:03:49.712504 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.712470 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b43b5f7-6748-4f95-90b1-ee0c8b49699d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qct59\" (UID: \"8b43b5f7-6748-4f95-90b1-ee0c8b49699d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" Jul 09 14:03:49.712654 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.712530 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz8nx\" (UniqueName: \"kubernetes.io/projected/8b43b5f7-6748-4f95-90b1-ee0c8b49699d-kube-api-access-kz8nx\") pod \"cert-manager-operator-controller-manager-55c444bf86-qct59\" (UID: \"8b43b5f7-6748-4f95-90b1-ee0c8b49699d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" Jul 09 14:03:49.723231 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.723199 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59"] Jul 09 14:03:49.813827 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.813780 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz8nx\" (UniqueName: \"kubernetes.io/projected/8b43b5f7-6748-4f95-90b1-ee0c8b49699d-kube-api-access-kz8nx\") pod \"cert-manager-operator-controller-manager-55c444bf86-qct59\" (UID: \"8b43b5f7-6748-4f95-90b1-ee0c8b49699d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" Jul 09 14:03:49.814024 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.813902 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b43b5f7-6748-4f95-90b1-ee0c8b49699d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qct59\" (UID: \"8b43b5f7-6748-4f95-90b1-ee0c8b49699d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" Jul 09 14:03:49.814227 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.814212 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b43b5f7-6748-4f95-90b1-ee0c8b49699d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-qct59\" (UID: \"8b43b5f7-6748-4f95-90b1-ee0c8b49699d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" Jul 09 14:03:49.836870 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:49.836800 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz8nx\" (UniqueName: \"kubernetes.io/projected/8b43b5f7-6748-4f95-90b1-ee0c8b49699d-kube-api-access-kz8nx\") pod \"cert-manager-operator-controller-manager-55c444bf86-qct59\" (UID: \"8b43b5f7-6748-4f95-90b1-ee0c8b49699d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" Jul 09 14:03:50.001020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:50.000923 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" Jul 09 14:03:50.230184 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:50.230154 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59"] Jul 09 14:03:50.232365 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:03:50.232337 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b43b5f7_6748_4f95_90b1_ee0c8b49699d.slice/crio-4a3a2beca99332dd6fe255310c7f27d35225bb8553e7c767b58d8fa5077816c4 WatchSource:0}: Error finding container 4a3a2beca99332dd6fe255310c7f27d35225bb8553e7c767b58d8fa5077816c4: Status 404 returned error can't find the container with id 4a3a2beca99332dd6fe255310c7f27d35225bb8553e7c767b58d8fa5077816c4 Jul 09 14:03:50.387274 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:50.387233 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" event={"ID":"8b43b5f7-6748-4f95-90b1-ee0c8b49699d","Type":"ContainerStarted","Data":"4a3a2beca99332dd6fe255310c7f27d35225bb8553e7c767b58d8fa5077816c4"} Jul 09 14:03:53.402706 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:53.402667 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" event={"ID":"8b43b5f7-6748-4f95-90b1-ee0c8b49699d","Type":"ContainerStarted","Data":"76ba7822bf8487bae350a3ec78d804685f5ba68c566fd89d75de86fb8a79b267"} Jul 09 14:03:53.443535 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:53.443477 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-qct59" podStartSLOduration=2.186240923 podStartE2EDuration="4.443459029s" totalImagesPullingTime="2.257218106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:53.441270662 +0000 UTC m=+497.295222140" watchObservedRunningTime="2026-07-09 14:03:53.443459029 +0000 UTC m=+497.297410504" Jul 09 14:03:54.391128 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.391091 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr"] Jul 09 14:03:54.393551 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.393533 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.415351 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.415321 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 14:03:54.416835 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.416820 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gmwc2\"" Jul 09 14:03:54.417573 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.417559 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 14:03:54.457619 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.457585 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr"] Jul 09 14:03:54.457928 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.457903 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.458053 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.458037 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-kube-api-access-nm7v9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.458160 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.458134 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.559310 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.559271 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.559512 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.559337 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.559512 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.559385 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-kube-api-access-nm7v9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.559729 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.559707 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.559792 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.559741 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.587219 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.587189 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-kube-api-access-nm7v9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.703016 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.702925 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:54.871646 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:54.871485 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr"] Jul 09 14:03:54.873616 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:03:54.873585 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71929acf_8c71_4f8d_9d86_34c1c0cbfd58.slice/crio-9c8a85828006390b7e37dda3aec3bbb66142dee60ecd0df5fcb3906fb8846868 WatchSource:0}: Error finding container 9c8a85828006390b7e37dda3aec3bbb66142dee60ecd0df5fcb3906fb8846868: Status 404 returned error can't find the container with id 9c8a85828006390b7e37dda3aec3bbb66142dee60ecd0df5fcb3906fb8846868 Jul 09 14:03:55.416998 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:55.416958 2575 generic.go:356] "Generic (PLEG): container finished" podID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerID="f2203ac09c1032e9fc0d7f6f689133c98432cb5ea4ae586300a0e58690fb9a7d" exitCode=0 Jul 09 14:03:55.417439 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:55.417011 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" event={"ID":"71929acf-8c71-4f8d-9d86-34c1c0cbfd58","Type":"ContainerDied","Data":"f2203ac09c1032e9fc0d7f6f689133c98432cb5ea4ae586300a0e58690fb9a7d"} Jul 09 14:03:55.417439 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:55.417088 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" event={"ID":"71929acf-8c71-4f8d-9d86-34c1c0cbfd58","Type":"ContainerStarted","Data":"9c8a85828006390b7e37dda3aec3bbb66142dee60ecd0df5fcb3906fb8846868"} Jul 09 14:03:57.428429 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:57.428386 2575 generic.go:356] "Generic (PLEG): container finished" podID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerID="89a2dcd0462a1cff10be4e4f7e9ae1b36d451079e9b8771b67023959bff3123f" exitCode=0 Jul 09 14:03:57.428921 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:57.428473 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" event={"ID":"71929acf-8c71-4f8d-9d86-34c1c0cbfd58","Type":"ContainerDied","Data":"89a2dcd0462a1cff10be4e4f7e9ae1b36d451079e9b8771b67023959bff3123f"} Jul 09 14:03:58.434515 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:58.434477 2575 generic.go:356] "Generic (PLEG): container finished" podID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerID="d16140d201a9c06ce57ec95dcde53c74e5e9c58b57e7ec7f30d21ee2a38b5f4e" exitCode=0 Jul 09 14:03:58.434942 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:58.434562 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" event={"ID":"71929acf-8c71-4f8d-9d86-34c1c0cbfd58","Type":"ContainerDied","Data":"d16140d201a9c06ce57ec95dcde53c74e5e9c58b57e7ec7f30d21ee2a38b5f4e"} Jul 09 14:03:59.563774 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.563749 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:03:59.605629 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.605592 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-util\") pod \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " Jul 09 14:03:59.605829 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.605651 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-bundle\") pod \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " Jul 09 14:03:59.605829 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.605715 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-kube-api-access-nm7v9\") pod \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\" (UID: \"71929acf-8c71-4f8d-9d86-34c1c0cbfd58\") " Jul 09 14:03:59.606131 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.606098 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-bundle" (OuterVolumeSpecName: "bundle") pod "71929acf-8c71-4f8d-9d86-34c1c0cbfd58" (UID: "71929acf-8c71-4f8d-9d86-34c1c0cbfd58"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:03:59.608293 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.608258 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-kube-api-access-nm7v9" (OuterVolumeSpecName: "kube-api-access-nm7v9") pod "71929acf-8c71-4f8d-9d86-34c1c0cbfd58" (UID: "71929acf-8c71-4f8d-9d86-34c1c0cbfd58"). InnerVolumeSpecName "kube-api-access-nm7v9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:03:59.611223 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.611190 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-util" (OuterVolumeSpecName: "util") pod "71929acf-8c71-4f8d-9d86-34c1c0cbfd58" (UID: "71929acf-8c71-4f8d-9d86-34c1c0cbfd58"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:03:59.706631 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.706541 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-util\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:03:59.706631 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.706568 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-bundle\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:03:59.706631 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:03:59.706578 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm7v9\" (UniqueName: \"kubernetes.io/projected/71929acf-8c71-4f8d-9d86-34c1c0cbfd58-kube-api-access-nm7v9\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:04:00.444604 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:00.444576 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" Jul 09 14:04:00.444781 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:00.444576 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7lxnr" event={"ID":"71929acf-8c71-4f8d-9d86-34c1c0cbfd58","Type":"ContainerDied","Data":"9c8a85828006390b7e37dda3aec3bbb66142dee60ecd0df5fcb3906fb8846868"} Jul 09 14:04:00.444781 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:00.444701 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c8a85828006390b7e37dda3aec3bbb66142dee60ecd0df5fcb3906fb8846868" Jul 09 14:04:01.689354 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.689311 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9m72h"] Jul 09 14:04:01.689841 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.689802 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerName="pull" Jul 09 14:04:01.689841 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.689820 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerName="pull" Jul 09 14:04:01.689841 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.689838 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerName="extract" Jul 09 14:04:01.689841 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.689867 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerName="extract" Jul 09 14:04:01.690096 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.689889 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerName="util" Jul 09 14:04:01.690096 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.689897 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerName="util" Jul 09 14:04:01.690096 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.689993 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="71929acf-8c71-4f8d-9d86-34c1c0cbfd58" containerName="extract" Jul 09 14:04:01.696872 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.693256 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" Jul 09 14:04:01.698726 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.698694 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ks5br\"" Jul 09 14:04:01.698886 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.698710 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 14:04:01.699822 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.699801 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 14:04:01.703106 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.703079 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9m72h"] Jul 09 14:04:01.826206 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.826169 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpzvg\" (UniqueName: \"kubernetes.io/projected/a0f66b58-0bd3-44bb-97c0-fdc61733c544-kube-api-access-wpzvg\") pod \"cert-manager-cainjector-559cdd8c87-9m72h\" (UID: \"a0f66b58-0bd3-44bb-97c0-fdc61733c544\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" Jul 09 14:04:01.826393 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.826240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0f66b58-0bd3-44bb-97c0-fdc61733c544-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9m72h\" (UID: \"a0f66b58-0bd3-44bb-97c0-fdc61733c544\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" Jul 09 14:04:01.926898 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.926837 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0f66b58-0bd3-44bb-97c0-fdc61733c544-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9m72h\" (UID: \"a0f66b58-0bd3-44bb-97c0-fdc61733c544\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" Jul 09 14:04:01.927096 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.926927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpzvg\" (UniqueName: \"kubernetes.io/projected/a0f66b58-0bd3-44bb-97c0-fdc61733c544-kube-api-access-wpzvg\") pod \"cert-manager-cainjector-559cdd8c87-9m72h\" (UID: \"a0f66b58-0bd3-44bb-97c0-fdc61733c544\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" Jul 09 14:04:01.935428 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.935392 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a0f66b58-0bd3-44bb-97c0-fdc61733c544-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9m72h\" (UID: \"a0f66b58-0bd3-44bb-97c0-fdc61733c544\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" Jul 09 14:04:01.935549 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:01.935539 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpzvg\" (UniqueName: \"kubernetes.io/projected/a0f66b58-0bd3-44bb-97c0-fdc61733c544-kube-api-access-wpzvg\") pod \"cert-manager-cainjector-559cdd8c87-9m72h\" (UID: \"a0f66b58-0bd3-44bb-97c0-fdc61733c544\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" Jul 09 14:04:02.020147 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:02.020117 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" Jul 09 14:04:02.160765 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:04:02.160734 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0f66b58_0bd3_44bb_97c0_fdc61733c544.slice/crio-0bbdc3cd805ffb5d0a5a79a67c08f0c26a9995db9160be18f07ce8a436d4f1f3 WatchSource:0}: Error finding container 0bbdc3cd805ffb5d0a5a79a67c08f0c26a9995db9160be18f07ce8a436d4f1f3: Status 404 returned error can't find the container with id 0bbdc3cd805ffb5d0a5a79a67c08f0c26a9995db9160be18f07ce8a436d4f1f3 Jul 09 14:04:02.161346 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:02.161317 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9m72h"] Jul 09 14:04:02.455514 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:02.455429 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" event={"ID":"a0f66b58-0bd3-44bb-97c0-fdc61733c544","Type":"ContainerStarted","Data":"0bbdc3cd805ffb5d0a5a79a67c08f0c26a9995db9160be18f07ce8a436d4f1f3"} Jul 09 14:04:04.973763 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:04.973713 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nktnk"] Jul 09 14:04:04.977335 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:04.977313 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nktnk" Jul 09 14:04:04.980012 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:04.979943 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nnlwf\"" Jul 09 14:04:04.990141 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:04.990111 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nktnk"] Jul 09 14:04:05.052863 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:05.052821 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da0cec30-dfb7-4e03-8b24-572cb58a66d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-nktnk\" (UID: \"da0cec30-dfb7-4e03-8b24-572cb58a66d2\") " pod="cert-manager/cert-manager-55b4d954d5-nktnk" Jul 09 14:04:05.053051 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:05.052897 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt6lg\" (UniqueName: \"kubernetes.io/projected/da0cec30-dfb7-4e03-8b24-572cb58a66d2-kube-api-access-gt6lg\") pod \"cert-manager-55b4d954d5-nktnk\" (UID: \"da0cec30-dfb7-4e03-8b24-572cb58a66d2\") " pod="cert-manager/cert-manager-55b4d954d5-nktnk" Jul 09 14:04:05.157577 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:05.155384 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da0cec30-dfb7-4e03-8b24-572cb58a66d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-nktnk\" (UID: \"da0cec30-dfb7-4e03-8b24-572cb58a66d2\") " pod="cert-manager/cert-manager-55b4d954d5-nktnk" Jul 09 14:04:05.157577 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:05.155443 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt6lg\" (UniqueName: \"kubernetes.io/projected/da0cec30-dfb7-4e03-8b24-572cb58a66d2-kube-api-access-gt6lg\") pod \"cert-manager-55b4d954d5-nktnk\" (UID: \"da0cec30-dfb7-4e03-8b24-572cb58a66d2\") " pod="cert-manager/cert-manager-55b4d954d5-nktnk" Jul 09 14:04:05.165993 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:05.165959 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da0cec30-dfb7-4e03-8b24-572cb58a66d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-nktnk\" (UID: \"da0cec30-dfb7-4e03-8b24-572cb58a66d2\") " pod="cert-manager/cert-manager-55b4d954d5-nktnk" Jul 09 14:04:05.166161 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:05.165961 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt6lg\" (UniqueName: \"kubernetes.io/projected/da0cec30-dfb7-4e03-8b24-572cb58a66d2-kube-api-access-gt6lg\") pod \"cert-manager-55b4d954d5-nktnk\" (UID: \"da0cec30-dfb7-4e03-8b24-572cb58a66d2\") " pod="cert-manager/cert-manager-55b4d954d5-nktnk" Jul 09 14:04:05.288987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:05.288957 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nktnk" Jul 09 14:04:05.452133 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:05.452097 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nktnk"] Jul 09 14:04:05.455306 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:04:05.455272 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda0cec30_dfb7_4e03_8b24_572cb58a66d2.slice/crio-0d80dce243445b82b7d61d035767185cc55cedf50a69d91d554c28ecc4f1496e WatchSource:0}: Error finding container 0d80dce243445b82b7d61d035767185cc55cedf50a69d91d554c28ecc4f1496e: Status 404 returned error can't find the container with id 0d80dce243445b82b7d61d035767185cc55cedf50a69d91d554c28ecc4f1496e Jul 09 14:04:05.470564 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:05.470523 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nktnk" event={"ID":"da0cec30-dfb7-4e03-8b24-572cb58a66d2","Type":"ContainerStarted","Data":"0d80dce243445b82b7d61d035767185cc55cedf50a69d91d554c28ecc4f1496e"} Jul 09 14:04:06.921997 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:06.921959 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b"] Jul 09 14:04:06.943487 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:06.943452 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b"] Jul 09 14:04:06.943636 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:06.943569 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" Jul 09 14:04:06.947234 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:06.947211 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 14:04:06.948478 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:06.948335 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gb92h\"" Jul 09 14:04:06.948478 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:06.948363 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 14:04:07.076964 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:07.076932 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlqkl\" (UniqueName: \"kubernetes.io/projected/212df6b7-3466-4c4c-84ef-071e017395cd-kube-api-access-qlqkl\") pod \"openshift-lws-operator-fd8ccff4c-ljf8b\" (UID: \"212df6b7-3466-4c4c-84ef-071e017395cd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" Jul 09 14:04:07.077140 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:07.077016 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/212df6b7-3466-4c4c-84ef-071e017395cd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ljf8b\" (UID: \"212df6b7-3466-4c4c-84ef-071e017395cd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" Jul 09 14:04:07.178210 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:07.178114 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlqkl\" (UniqueName: \"kubernetes.io/projected/212df6b7-3466-4c4c-84ef-071e017395cd-kube-api-access-qlqkl\") pod \"openshift-lws-operator-fd8ccff4c-ljf8b\" (UID: \"212df6b7-3466-4c4c-84ef-071e017395cd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" Jul 09 14:04:07.178210 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:07.178197 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/212df6b7-3466-4c4c-84ef-071e017395cd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ljf8b\" (UID: \"212df6b7-3466-4c4c-84ef-071e017395cd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" Jul 09 14:04:07.178517 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:07.178501 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/212df6b7-3466-4c4c-84ef-071e017395cd-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ljf8b\" (UID: \"212df6b7-3466-4c4c-84ef-071e017395cd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" Jul 09 14:04:07.187967 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:07.187940 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlqkl\" (UniqueName: \"kubernetes.io/projected/212df6b7-3466-4c4c-84ef-071e017395cd-kube-api-access-qlqkl\") pod \"openshift-lws-operator-fd8ccff4c-ljf8b\" (UID: \"212df6b7-3466-4c4c-84ef-071e017395cd\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" Jul 09 14:04:07.253103 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:07.253066 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" Jul 09 14:04:07.507731 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:07.507706 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b"] Jul 09 14:04:07.510768 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:04:07.510741 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod212df6b7_3466_4c4c_84ef_071e017395cd.slice/crio-64084b75e60e1087a7af29d49843ab8b583585418a8457dbfe87669e7c9bc648 WatchSource:0}: Error finding container 64084b75e60e1087a7af29d49843ab8b583585418a8457dbfe87669e7c9bc648: Status 404 returned error can't find the container with id 64084b75e60e1087a7af29d49843ab8b583585418a8457dbfe87669e7c9bc648 Jul 09 14:04:08.487335 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:08.487291 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" event={"ID":"212df6b7-3466-4c4c-84ef-071e017395cd","Type":"ContainerStarted","Data":"64084b75e60e1087a7af29d49843ab8b583585418a8457dbfe87669e7c9bc648"} Jul 09 14:04:08.489219 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:08.489182 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nktnk" event={"ID":"da0cec30-dfb7-4e03-8b24-572cb58a66d2","Type":"ContainerStarted","Data":"c9549f9d86691a34e2b5999fda41ebc2a1340e2cd2e2b83e178791c4f46ad35a"} Jul 09 14:04:08.491028 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:08.491003 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" event={"ID":"a0f66b58-0bd3-44bb-97c0-fdc61733c544","Type":"ContainerStarted","Data":"600f1b51cbe835a33a4169f201ea3d7d60c5c13816da9f3734aeb24bcb6669b5"} Jul 09 14:04:08.511838 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:08.511668 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-nktnk" podStartSLOduration=2.604178249 podStartE2EDuration="4.511645795s" totalImagesPullingTime="1.907467546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:08.510405675 +0000 UTC m=+512.364357152" watchObservedRunningTime="2026-07-09 14:04:08.511645795 +0000 UTC m=+512.365597272" Jul 09 14:04:08.533147 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:08.533077 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9m72h" podStartSLOduration=2.322889067 podStartE2EDuration="7.533053128s" totalImagesPullingTime="5.210164061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:08.530716065 +0000 UTC m=+512.384667542" watchObservedRunningTime="2026-07-09 14:04:08.533053128 +0000 UTC m=+512.387004605" Jul 09 14:04:10.503934 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:10.503898 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" event={"ID":"212df6b7-3466-4c4c-84ef-071e017395cd","Type":"ContainerStarted","Data":"e1cbdecbeee82d64888d63e693aba580eafd73bb470687322b10c64123b72821"} Jul 09 14:04:10.522162 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:10.522104 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ljf8b" podStartSLOduration=2.205785183 podStartE2EDuration="4.522085211s" totalImagesPullingTime="2.316300028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:10.520665866 +0000 UTC m=+514.374617354" watchObservedRunningTime="2026-07-09 14:04:10.522085211 +0000 UTC m=+514.376036736" Jul 09 14:04:12.943881 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:12.943830 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29"] Jul 09 14:04:12.946406 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:12.946389 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:12.949092 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:12.949070 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 14:04:12.950120 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:12.950088 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gmwc2\"" Jul 09 14:04:12.950120 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:12.950088 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 14:04:12.959506 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:12.959480 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29"] Jul 09 14:04:13.028289 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.028254 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdrpw\" (UniqueName: \"kubernetes.io/projected/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-kube-api-access-fdrpw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.028479 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.028305 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.028479 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.028340 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.129674 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.129635 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdrpw\" (UniqueName: \"kubernetes.io/projected/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-kube-api-access-fdrpw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.129911 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.129690 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.129911 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.129812 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.130075 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.130057 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.130133 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.130115 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.139242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.139210 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdrpw\" (UniqueName: \"kubernetes.io/projected/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-kube-api-access-fdrpw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.255712 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.255623 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:13.395236 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.395176 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29"] Jul 09 14:04:13.397468 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:04:13.397442 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode217f321_b898_4a5e_b4e8_1905ab7bc5ef.slice/crio-5dbed052db49d68858314e67885ad041f992546ade06f9741d4e408df8bcf738 WatchSource:0}: Error finding container 5dbed052db49d68858314e67885ad041f992546ade06f9741d4e408df8bcf738: Status 404 returned error can't find the container with id 5dbed052db49d68858314e67885ad041f992546ade06f9741d4e408df8bcf738 Jul 09 14:04:13.519264 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.519228 2575 generic.go:356] "Generic (PLEG): container finished" podID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerID="ab6beda65a8536a75d2985b9a01644046a7fabb4918cf8a24eafa15e1d5fa60e" exitCode=0 Jul 09 14:04:13.519440 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.519321 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" event={"ID":"e217f321-b898-4a5e-b4e8-1905ab7bc5ef","Type":"ContainerDied","Data":"ab6beda65a8536a75d2985b9a01644046a7fabb4918cf8a24eafa15e1d5fa60e"} Jul 09 14:04:13.519440 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:13.519376 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" event={"ID":"e217f321-b898-4a5e-b4e8-1905ab7bc5ef","Type":"ContainerStarted","Data":"5dbed052db49d68858314e67885ad041f992546ade06f9741d4e408df8bcf738"} Jul 09 14:04:14.526295 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:14.526262 2575 generic.go:356] "Generic (PLEG): container finished" podID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerID="509816b8df627233163d8de7b37cf29b4a0d63525c98e98b436abc39802fb5b5" exitCode=0 Jul 09 14:04:14.526799 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:14.526355 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" event={"ID":"e217f321-b898-4a5e-b4e8-1905ab7bc5ef","Type":"ContainerDied","Data":"509816b8df627233163d8de7b37cf29b4a0d63525c98e98b436abc39802fb5b5"} Jul 09 14:04:15.533284 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:15.533247 2575 generic.go:356] "Generic (PLEG): container finished" podID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerID="e0bea7e81daa3ac319be4d08cb0e7a44d8c983bb4bf7fe2167a95a7d4e902ae9" exitCode=0 Jul 09 14:04:15.533692 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:15.533374 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" event={"ID":"e217f321-b898-4a5e-b4e8-1905ab7bc5ef","Type":"ContainerDied","Data":"e0bea7e81daa3ac319be4d08cb0e7a44d8c983bb4bf7fe2167a95a7d4e902ae9"} Jul 09 14:04:16.671044 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.671015 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:16.761135 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.761099 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdrpw\" (UniqueName: \"kubernetes.io/projected/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-kube-api-access-fdrpw\") pod \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " Jul 09 14:04:16.761308 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.761193 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-bundle\") pod \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " Jul 09 14:04:16.761308 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.761236 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-util\") pod \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\" (UID: \"e217f321-b898-4a5e-b4e8-1905ab7bc5ef\") " Jul 09 14:04:16.761969 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.761945 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-bundle" (OuterVolumeSpecName: "bundle") pod "e217f321-b898-4a5e-b4e8-1905ab7bc5ef" (UID: "e217f321-b898-4a5e-b4e8-1905ab7bc5ef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:04:16.763353 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.763328 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-kube-api-access-fdrpw" (OuterVolumeSpecName: "kube-api-access-fdrpw") pod "e217f321-b898-4a5e-b4e8-1905ab7bc5ef" (UID: "e217f321-b898-4a5e-b4e8-1905ab7bc5ef"). InnerVolumeSpecName "kube-api-access-fdrpw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:04:16.766196 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.766173 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-util" (OuterVolumeSpecName: "util") pod "e217f321-b898-4a5e-b4e8-1905ab7bc5ef" (UID: "e217f321-b898-4a5e-b4e8-1905ab7bc5ef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:04:16.862231 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.862144 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdrpw\" (UniqueName: \"kubernetes.io/projected/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-kube-api-access-fdrpw\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:04:16.862231 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.862173 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-bundle\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:04:16.862231 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:16.862185 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e217f321-b898-4a5e-b4e8-1905ab7bc5ef-util\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:04:17.544689 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:17.544659 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" Jul 09 14:04:17.544689 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:17.544680 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhrx29" event={"ID":"e217f321-b898-4a5e-b4e8-1905ab7bc5ef","Type":"ContainerDied","Data":"5dbed052db49d68858314e67885ad041f992546ade06f9741d4e408df8bcf738"} Jul 09 14:04:17.544922 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:17.544726 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5dbed052db49d68858314e67885ad041f992546ade06f9741d4e408df8bcf738" Jul 09 14:04:26.038378 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.038340 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d"] Jul 09 14:04:26.038787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.038665 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerName="pull" Jul 09 14:04:26.038787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.038676 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerName="pull" Jul 09 14:04:26.038787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.038689 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerName="extract" Jul 09 14:04:26.038787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.038694 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerName="extract" Jul 09 14:04:26.038787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.038700 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerName="util" Jul 09 14:04:26.038787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.038706 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerName="util" Jul 09 14:04:26.038787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.038765 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e217f321-b898-4a5e-b4e8-1905ab7bc5ef" containerName="extract" Jul 09 14:04:26.042310 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.042292 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.044869 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.044827 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-djqn2\"" Jul 09 14:04:26.044978 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.044932 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 14:04:26.045482 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.045467 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 14:04:26.045956 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.045935 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 14:04:26.052489 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.052466 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 14:04:26.070697 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.070664 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d"] Jul 09 14:04:26.147610 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.147577 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b28d7a4-503a-4a59-85c1-c433b1776eb8-manifests\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.147610 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.147628 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b28d7a4-503a-4a59-85c1-c433b1776eb8-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.147989 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.147732 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b28d7a4-503a-4a59-85c1-c433b1776eb8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.147989 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.147798 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vtwm\" (UniqueName: \"kubernetes.io/projected/1b28d7a4-503a-4a59-85c1-c433b1776eb8-kube-api-access-2vtwm\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.147989 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.147873 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b28d7a4-503a-4a59-85c1-c433b1776eb8-tmp\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.248915 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.248863 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b28d7a4-503a-4a59-85c1-c433b1776eb8-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.249102 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.248958 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b28d7a4-503a-4a59-85c1-c433b1776eb8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.249102 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.249019 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtwm\" (UniqueName: \"kubernetes.io/projected/1b28d7a4-503a-4a59-85c1-c433b1776eb8-kube-api-access-2vtwm\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.249102 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.249065 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b28d7a4-503a-4a59-85c1-c433b1776eb8-tmp\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.249296 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.249132 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b28d7a4-503a-4a59-85c1-c433b1776eb8-manifests\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.249626 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.249602 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b28d7a4-503a-4a59-85c1-c433b1776eb8-manifests\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.251802 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.251771 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b28d7a4-503a-4a59-85c1-c433b1776eb8-tmp\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.251943 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.251799 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b28d7a4-503a-4a59-85c1-c433b1776eb8-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.252014 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.251967 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b28d7a4-503a-4a59-85c1-c433b1776eb8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.262263 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.262233 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vtwm\" (UniqueName: \"kubernetes.io/projected/1b28d7a4-503a-4a59-85c1-c433b1776eb8-kube-api-access-2vtwm\") pod \"opendatahub-operator-controller-manager-5d74b946fd-p6m9d\" (UID: \"1b28d7a4-503a-4a59-85c1-c433b1776eb8\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.353747 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.353657 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:26.507272 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.507244 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d"] Jul 09 14:04:26.508820 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:04:26.508795 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b28d7a4_503a_4a59_85c1_c433b1776eb8.slice/crio-aafaeb8cfaf90b204a9dbd70ba4cae6f8d6031963f7f6d3d4c9e70e0eb066d84 WatchSource:0}: Error finding container aafaeb8cfaf90b204a9dbd70ba4cae6f8d6031963f7f6d3d4c9e70e0eb066d84: Status 404 returned error can't find the container with id aafaeb8cfaf90b204a9dbd70ba4cae6f8d6031963f7f6d3d4c9e70e0eb066d84 Jul 09 14:04:26.593448 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:26.593406 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" event={"ID":"1b28d7a4-503a-4a59-85c1-c433b1776eb8","Type":"ContainerStarted","Data":"aafaeb8cfaf90b204a9dbd70ba4cae6f8d6031963f7f6d3d4c9e70e0eb066d84"} Jul 09 14:04:29.612121 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:29.612072 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" event={"ID":"1b28d7a4-503a-4a59-85c1-c433b1776eb8","Type":"ContainerStarted","Data":"1721dad5812f2223b18dbc7beb248003cff2662cc42b78be1a0eb198e68b6e05"} Jul 09 14:04:30.619303 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:30.619269 2575 generic.go:356] "Generic (PLEG): container finished" podID="1b28d7a4-503a-4a59-85c1-c433b1776eb8" containerID="1721dad5812f2223b18dbc7beb248003cff2662cc42b78be1a0eb198e68b6e05" exitCode=0 Jul 09 14:04:30.619686 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:30.619347 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" event={"ID":"1b28d7a4-503a-4a59-85c1-c433b1776eb8","Type":"ContainerDied","Data":"1721dad5812f2223b18dbc7beb248003cff2662cc42b78be1a0eb198e68b6e05"} Jul 09 14:04:31.626838 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:31.626804 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" event={"ID":"1b28d7a4-503a-4a59-85c1-c433b1776eb8","Type":"ContainerStarted","Data":"ede3c3e53c137c97337bc77d7a139592379527d8d9c745fce24e863712b4de62"} Jul 09 14:04:31.627259 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:31.626915 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:31.657657 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:31.657580 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" podStartSLOduration=1.227381522 podStartE2EDuration="5.657554391s" totalImagesPullingTime="3.418670206s" totalInitContainerRuntime="1.011502663s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:31.653961435 +0000 UTC m=+535.507912914" watchObservedRunningTime="2026-07-09 14:04:31.657554391 +0000 UTC m=+535.511505869" Jul 09 14:04:42.637285 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:42.637259 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d74b946fd-p6m9d" Jul 09 14:04:44.065037 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.065000 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g"] Jul 09 14:04:44.074318 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.074285 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.077820 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.077793 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g"] Jul 09 14:04:44.078154 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.078135 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 14:04:44.078225 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.078153 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8zgfx\"" Jul 09 14:04:44.078284 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.078264 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 14:04:44.078413 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.078391 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 14:04:44.213870 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.213816 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/53cfee3e-a549-4293-86e1-5dfb65a76cf6-metrics-cert\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.214062 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.213890 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/53cfee3e-a549-4293-86e1-5dfb65a76cf6-manager-config\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.214062 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.214026 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9bg8\" (UniqueName: \"kubernetes.io/projected/53cfee3e-a549-4293-86e1-5dfb65a76cf6-kube-api-access-n9bg8\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.214172 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.214147 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53cfee3e-a549-4293-86e1-5dfb65a76cf6-cert\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.315013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.314970 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53cfee3e-a549-4293-86e1-5dfb65a76cf6-cert\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.315013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.315019 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/53cfee3e-a549-4293-86e1-5dfb65a76cf6-metrics-cert\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.315319 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.315047 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/53cfee3e-a549-4293-86e1-5dfb65a76cf6-manager-config\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.315319 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.315083 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9bg8\" (UniqueName: \"kubernetes.io/projected/53cfee3e-a549-4293-86e1-5dfb65a76cf6-kube-api-access-n9bg8\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.315793 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.315767 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/53cfee3e-a549-4293-86e1-5dfb65a76cf6-manager-config\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.317631 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.317608 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53cfee3e-a549-4293-86e1-5dfb65a76cf6-cert\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.317730 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.317678 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/53cfee3e-a549-4293-86e1-5dfb65a76cf6-metrics-cert\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.329428 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.329398 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9bg8\" (UniqueName: \"kubernetes.io/projected/53cfee3e-a549-4293-86e1-5dfb65a76cf6-kube-api-access-n9bg8\") pod \"lws-controller-manager-c98bd85cf-5677g\" (UID: \"53cfee3e-a549-4293-86e1-5dfb65a76cf6\") " pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.385247 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.385207 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:44.527063 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.527037 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g"] Jul 09 14:04:44.529433 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:04:44.529402 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53cfee3e_a549_4293_86e1_5dfb65a76cf6.slice/crio-1d390e42c20530a92fcaa912f7aa7425f4b90e467004c5918e8c040d75d17ee4 WatchSource:0}: Error finding container 1d390e42c20530a92fcaa912f7aa7425f4b90e467004c5918e8c040d75d17ee4: Status 404 returned error can't find the container with id 1d390e42c20530a92fcaa912f7aa7425f4b90e467004c5918e8c040d75d17ee4 Jul 09 14:04:44.694310 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:44.694228 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" event={"ID":"53cfee3e-a549-4293-86e1-5dfb65a76cf6","Type":"ContainerStarted","Data":"1d390e42c20530a92fcaa912f7aa7425f4b90e467004c5918e8c040d75d17ee4"} Jul 09 14:04:47.712178 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:47.712085 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" event={"ID":"53cfee3e-a549-4293-86e1-5dfb65a76cf6","Type":"ContainerStarted","Data":"8419edeb14924b48d3156947127c9273f23281288365b9fcd8b3842aafbd627e"} Jul 09 14:04:47.712592 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:47.712218 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:47.736727 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:47.736670 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" podStartSLOduration=0.875329355 podStartE2EDuration="3.736651654s" totalImagesPullingTime="2.861322299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:47.735231088 +0000 UTC m=+551.589182563" watchObservedRunningTime="2026-07-09 14:04:47.736651654 +0000 UTC m=+551.590603141" Jul 09 14:04:54.045398 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.045357 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g"] Jul 09 14:04:54.050322 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.050299 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.053913 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.053875 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qrktm\"" Jul 09 14:04:54.054995 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.054975 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 14:04:54.075007 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.074970 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g"] Jul 09 14:04:54.208589 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.208544 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.208806 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.208597 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.208806 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.208705 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c20d428-334e-4a19-905a-67144babe143-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.208806 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.208772 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c20d428-334e-4a19-905a-67144babe143-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.209033 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.208836 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.209033 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.208912 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hcpk\" (UniqueName: \"kubernetes.io/projected/2c20d428-334e-4a19-905a-67144babe143-kube-api-access-8hcpk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.209033 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.208952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.209033 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.209016 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c20d428-334e-4a19-905a-67144babe143-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.209167 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.209068 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.309837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.309734 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c20d428-334e-4a19-905a-67144babe143-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.309837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.309778 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.309837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.309811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.309837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.309829 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.309896 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c20d428-334e-4a19-905a-67144babe143-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.309938 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c20d428-334e-4a19-905a-67144babe143-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.309980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.310026 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hcpk\" (UniqueName: \"kubernetes.io/projected/2c20d428-334e-4a19-905a-67144babe143-kube-api-access-8hcpk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.310062 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310463 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.310426 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310583 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.310547 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310665 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.310588 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310708 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.310661 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.310787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.310768 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c20d428-334e-4a19-905a-67144babe143-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.312500 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.312479 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c20d428-334e-4a19-905a-67144babe143-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.312919 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.312893 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c20d428-334e-4a19-905a-67144babe143-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.321436 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.321394 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c20d428-334e-4a19-905a-67144babe143-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.321631 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.321614 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hcpk\" (UniqueName: \"kubernetes.io/projected/2c20d428-334e-4a19-905a-67144babe143-kube-api-access-8hcpk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h866g\" (UID: \"2c20d428-334e-4a19-905a-67144babe143\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.363610 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.363571 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:54.527655 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.527629 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g"] Jul 09 14:04:54.529697 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:04:54.529664 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c20d428_334e_4a19_905a_67144babe143.slice/crio-cf33203548126ccbc7596d7f20ff49038949bfe5d4c4595563d7e2806699e3f4 WatchSource:0}: Error finding container cf33203548126ccbc7596d7f20ff49038949bfe5d4c4595563d7e2806699e3f4: Status 404 returned error can't find the container with id cf33203548126ccbc7596d7f20ff49038949bfe5d4c4595563d7e2806699e3f4 Jul 09 14:04:54.747625 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:54.747530 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" event={"ID":"2c20d428-334e-4a19-905a-67144babe143","Type":"ContainerStarted","Data":"cf33203548126ccbc7596d7f20ff49038949bfe5d4c4595563d7e2806699e3f4"} Jul 09 14:04:57.478166 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.478117 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:04:57.478552 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.478216 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:04:57.478552 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.478266 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:04:57.767639 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.767602 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" event={"ID":"2c20d428-334e-4a19-905a-67144babe143","Type":"ContainerStarted","Data":"fb7fda4a4716f04134d774eef915ec2812b0d172693dc2d1f96ed713cdf89b5c"} Jul 09 14:04:57.837607 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.837540 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" podStartSLOduration=0.891793631 podStartE2EDuration="3.837517341s" totalImagesPullingTime="2.94572371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:57.835106681 +0000 UTC m=+561.689058158" watchObservedRunningTime="2026-07-09 14:04:57.837517341 +0000 UTC m=+561.691468829" Jul 09 14:04:57.878842 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.878809 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq"] Jul 09 14:04:57.882504 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.882486 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:04:57.887223 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.887200 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-hdvkt\"" Jul 09 14:04:57.892527 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.892501 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 14:04:57.906400 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:57.906372 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq"] Jul 09 14:04:58.046714 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.046625 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/859a3a7e-f0b7-4395-9c82-ee497dd62aba-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wpzxq\" (UID: \"859a3a7e-f0b7-4395-9c82-ee497dd62aba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:04:58.046714 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.046684 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8cbf\" (UniqueName: \"kubernetes.io/projected/859a3a7e-f0b7-4395-9c82-ee497dd62aba-kube-api-access-k8cbf\") pod \"llmisvc-controller-manager-777c6bd7d5-wpzxq\" (UID: \"859a3a7e-f0b7-4395-9c82-ee497dd62aba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:04:58.148292 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.148249 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/859a3a7e-f0b7-4395-9c82-ee497dd62aba-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wpzxq\" (UID: \"859a3a7e-f0b7-4395-9c82-ee497dd62aba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:04:58.148494 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.148320 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8cbf\" (UniqueName: \"kubernetes.io/projected/859a3a7e-f0b7-4395-9c82-ee497dd62aba-kube-api-access-k8cbf\") pod \"llmisvc-controller-manager-777c6bd7d5-wpzxq\" (UID: \"859a3a7e-f0b7-4395-9c82-ee497dd62aba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:04:58.150870 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.150832 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/859a3a7e-f0b7-4395-9c82-ee497dd62aba-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wpzxq\" (UID: \"859a3a7e-f0b7-4395-9c82-ee497dd62aba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:04:58.160580 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.160553 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8cbf\" (UniqueName: \"kubernetes.io/projected/859a3a7e-f0b7-4395-9c82-ee497dd62aba-kube-api-access-k8cbf\") pod \"llmisvc-controller-manager-777c6bd7d5-wpzxq\" (UID: \"859a3a7e-f0b7-4395-9c82-ee497dd62aba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:04:58.194110 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.194065 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:04:58.364155 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.364041 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:58.368581 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.368513 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq"] Jul 09 14:04:58.369717 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.369692 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:58.370887 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:04:58.370842 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod859a3a7e_f0b7_4395_9c82_ee497dd62aba.slice/crio-34fc02957ace4689ef06a8ed40ff08f5743c6f59cbfe33b29bc6de2cdd44883d WatchSource:0}: Error finding container 34fc02957ace4689ef06a8ed40ff08f5743c6f59cbfe33b29bc6de2cdd44883d: Status 404 returned error can't find the container with id 34fc02957ace4689ef06a8ed40ff08f5743c6f59cbfe33b29bc6de2cdd44883d Jul 09 14:04:58.720417 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.720329 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c98bd85cf-5677g" Jul 09 14:04:58.776004 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.775964 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" event={"ID":"859a3a7e-f0b7-4395-9c82-ee497dd62aba","Type":"ContainerStarted","Data":"34fc02957ace4689ef06a8ed40ff08f5743c6f59cbfe33b29bc6de2cdd44883d"} Jul 09 14:04:58.776600 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.776453 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:04:58.777728 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:04:58.777700 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h866g" Jul 09 14:05:01.792516 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:05:01.792481 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" event={"ID":"859a3a7e-f0b7-4395-9c82-ee497dd62aba","Type":"ContainerStarted","Data":"0924e5195dc2226da126163fef59c683bfb924745e929b60d4ffe875c4eaa46b"} Jul 09 14:05:01.792981 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:05:01.792597 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:05:01.833020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:05:01.832963 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" podStartSLOduration=2.082267262 podStartE2EDuration="4.832944064s" totalImagesPullingTime="2.750676802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:04:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:05:01.830747533 +0000 UTC m=+565.684699011" watchObservedRunningTime="2026-07-09 14:05:01.832944064 +0000 UTC m=+565.686895539" Jul 09 14:05:32.799547 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:05:32.799515 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wpzxq" Jul 09 14:06:02.618723 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.618624 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774"] Jul 09 14:06:02.621417 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.621396 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.624249 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.624218 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 14:06:02.624418 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.624405 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vh7wf\"" Jul 09 14:06:02.625437 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.625422 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 14:06:02.637809 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.637781 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774"] Jul 09 14:06:02.728386 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.728339 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k27rk\" (UniqueName: \"kubernetes.io/projected/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-kube-api-access-k27rk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.728564 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.728400 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.728564 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.728495 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.829702 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.829669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.829900 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.829723 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.829900 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.829775 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k27rk\" (UniqueName: \"kubernetes.io/projected/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-kube-api-access-k27rk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.830086 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.830068 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.830164 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.830147 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.840065 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.840031 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k27rk\" (UniqueName: \"kubernetes.io/projected/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-kube-api-access-k27rk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:02.930780 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:02.930698 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:03.067982 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.067950 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774"] Jul 09 14:06:03.069868 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:06:03.069828 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab40ba36_2ab1_41dc_a16e_4fc0f1f150c4.slice/crio-bd333b9fffc84d8d3242cc32b9bf25bee241b2e78a24dfd7b4489af0bbfa66ef WatchSource:0}: Error finding container bd333b9fffc84d8d3242cc32b9bf25bee241b2e78a24dfd7b4489af0bbfa66ef: Status 404 returned error can't find the container with id bd333b9fffc84d8d3242cc32b9bf25bee241b2e78a24dfd7b4489af0bbfa66ef Jul 09 14:06:03.093186 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.093150 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" event={"ID":"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4","Type":"ContainerStarted","Data":"bd333b9fffc84d8d3242cc32b9bf25bee241b2e78a24dfd7b4489af0bbfa66ef"} Jul 09 14:06:03.394957 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.394921 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx"] Jul 09 14:06:03.397923 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.397902 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.408164 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.408136 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx"] Jul 09 14:06:03.435052 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.435013 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj7md\" (UniqueName: \"kubernetes.io/projected/aaeb587b-9833-486c-a551-a8a1ee38a561-kube-api-access-lj7md\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.435052 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.435055 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.435278 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.435164 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.536362 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.536325 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj7md\" (UniqueName: \"kubernetes.io/projected/aaeb587b-9833-486c-a551-a8a1ee38a561-kube-api-access-lj7md\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.536362 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.536364 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.536605 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.536412 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.536797 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.536774 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.536914 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.536832 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.545981 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.545956 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj7md\" (UniqueName: \"kubernetes.io/projected/aaeb587b-9833-486c-a551-a8a1ee38a561-kube-api-access-lj7md\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.708420 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.708325 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:03.844890 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:03.844836 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx"] Jul 09 14:06:03.848389 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:06:03.848362 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaeb587b_9833_486c_a551_a8a1ee38a561.slice/crio-eef12f65fb10e16ab71867d5c2fadad221347882ec90016d0e8968eab7dcb4a6 WatchSource:0}: Error finding container eef12f65fb10e16ab71867d5c2fadad221347882ec90016d0e8968eab7dcb4a6: Status 404 returned error can't find the container with id eef12f65fb10e16ab71867d5c2fadad221347882ec90016d0e8968eab7dcb4a6 Jul 09 14:06:04.100392 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.100358 2575 generic.go:356] "Generic (PLEG): container finished" podID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerID="fe2ffe302a64f04991e7718c114a30caa571f2b0a7b5f449ef7fcac9995589f8" exitCode=0 Jul 09 14:06:04.100599 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.100447 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" event={"ID":"aaeb587b-9833-486c-a551-a8a1ee38a561","Type":"ContainerDied","Data":"fe2ffe302a64f04991e7718c114a30caa571f2b0a7b5f449ef7fcac9995589f8"} Jul 09 14:06:04.100599 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.100504 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" event={"ID":"aaeb587b-9833-486c-a551-a8a1ee38a561","Type":"ContainerStarted","Data":"eef12f65fb10e16ab71867d5c2fadad221347882ec90016d0e8968eab7dcb4a6"} Jul 09 14:06:04.100960 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.100936 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:06:04.102291 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.102273 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerID="abcb450a276f121a206c30145f52f4dbe5928f4ab08c2f7f580333d98530c551" exitCode=0 Jul 09 14:06:04.102379 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.102359 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" event={"ID":"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4","Type":"ContainerDied","Data":"abcb450a276f121a206c30145f52f4dbe5928f4ab08c2f7f580333d98530c551"} Jul 09 14:06:04.192363 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.192327 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr"] Jul 09 14:06:04.195905 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.195888 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.205503 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.205476 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr"] Jul 09 14:06:04.242769 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.242732 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klc47\" (UniqueName: \"kubernetes.io/projected/4152889b-5503-4d27-a0ec-33d4722e2652-kube-api-access-klc47\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.242966 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.242791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.242966 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.242887 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.344024 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.343972 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klc47\" (UniqueName: \"kubernetes.io/projected/4152889b-5503-4d27-a0ec-33d4722e2652-kube-api-access-klc47\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.344206 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.344104 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.344206 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.344158 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.344501 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.344482 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.344536 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.344517 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.354936 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.354879 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klc47\" (UniqueName: \"kubernetes.io/projected/4152889b-5503-4d27-a0ec-33d4722e2652-kube-api-access-klc47\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.505966 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.505934 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:04.691216 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.691187 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr"] Jul 09 14:06:04.693213 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:06:04.693181 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4152889b_5503_4d27_a0ec_33d4722e2652.slice/crio-0a938623a02b53c4339b3587ad89afcab68bc7e4f5bb8e0b6b3bd36c060082ff WatchSource:0}: Error finding container 0a938623a02b53c4339b3587ad89afcab68bc7e4f5bb8e0b6b3bd36c060082ff: Status 404 returned error can't find the container with id 0a938623a02b53c4339b3587ad89afcab68bc7e4f5bb8e0b6b3bd36c060082ff Jul 09 14:06:04.737135 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:04.737097 2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab40ba36_2ab1_41dc_a16e_4fc0f1f150c4.slice/crio-conmon-a593022414ff1bdaa61d6f0f08224791462fae0f510da6fd6ba4cd827de507c9.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab40ba36_2ab1_41dc_a16e_4fc0f1f150c4.slice/crio-a593022414ff1bdaa61d6f0f08224791462fae0f510da6fd6ba4cd827de507c9.scope\": RecentStats: unable to find data in memory cache]" Jul 09 14:06:04.800181 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.800142 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8"] Jul 09 14:06:04.803723 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.803702 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:04.814891 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.814861 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8"] Jul 09 14:06:04.848482 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.848451 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsqb4\" (UniqueName: \"kubernetes.io/projected/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-kube-api-access-xsqb4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:04.848612 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.848560 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:04.848612 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.848593 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:04.949370 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.949342 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsqb4\" (UniqueName: \"kubernetes.io/projected/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-kube-api-access-xsqb4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:04.949489 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.949413 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:04.949489 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.949443 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:04.949882 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.949830 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:04.950015 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.949886 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:04.959107 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:04.959079 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsqb4\" (UniqueName: \"kubernetes.io/projected/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-kube-api-access-xsqb4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:05.109629 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:05.109593 2575 generic.go:356] "Generic (PLEG): container finished" podID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerID="1bb48f77852a7a6c852385b19d620042db38bc316c9adf89b4648fb354a80a14" exitCode=0 Jul 09 14:06:05.109812 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:05.109682 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" event={"ID":"aaeb587b-9833-486c-a551-a8a1ee38a561","Type":"ContainerDied","Data":"1bb48f77852a7a6c852385b19d620042db38bc316c9adf89b4648fb354a80a14"} Jul 09 14:06:05.111384 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:05.111267 2575 generic.go:356] "Generic (PLEG): container finished" podID="4152889b-5503-4d27-a0ec-33d4722e2652" containerID="e36b5c194fe339a07fb1d650c42bad12b2078413d11c8364ceafdb7388bd0ae7" exitCode=0 Jul 09 14:06:05.111384 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:05.111346 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" event={"ID":"4152889b-5503-4d27-a0ec-33d4722e2652","Type":"ContainerDied","Data":"e36b5c194fe339a07fb1d650c42bad12b2078413d11c8364ceafdb7388bd0ae7"} Jul 09 14:06:05.111503 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:05.111397 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" event={"ID":"4152889b-5503-4d27-a0ec-33d4722e2652","Type":"ContainerStarted","Data":"0a938623a02b53c4339b3587ad89afcab68bc7e4f5bb8e0b6b3bd36c060082ff"} Jul 09 14:06:05.113252 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:05.113232 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerID="a593022414ff1bdaa61d6f0f08224791462fae0f510da6fd6ba4cd827de507c9" exitCode=0 Jul 09 14:06:05.113361 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:05.113319 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" event={"ID":"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4","Type":"ContainerDied","Data":"a593022414ff1bdaa61d6f0f08224791462fae0f510da6fd6ba4cd827de507c9"} Jul 09 14:06:05.115921 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:05.115906 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:05.287057 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:05.286993 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8"] Jul 09 14:06:05.325506 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:06:05.325466 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe992216_6c4e_4f24_9ab1_0cb62f7c8ee1.slice/crio-092b8e2140ff9cd3d28ff2a79e4954ddb46159ab51b60e52017a22ee8c7a2287 WatchSource:0}: Error finding container 092b8e2140ff9cd3d28ff2a79e4954ddb46159ab51b60e52017a22ee8c7a2287: Status 404 returned error can't find the container with id 092b8e2140ff9cd3d28ff2a79e4954ddb46159ab51b60e52017a22ee8c7a2287 Jul 09 14:06:06.120133 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:06.120098 2575 generic.go:356] "Generic (PLEG): container finished" podID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerID="12c9fc328eea6df33ad13fcd4cbe7f96956bbd1c6bf0c623f249265915025eb7" exitCode=0 Jul 09 14:06:06.120573 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:06.120166 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" event={"ID":"aaeb587b-9833-486c-a551-a8a1ee38a561","Type":"ContainerDied","Data":"12c9fc328eea6df33ad13fcd4cbe7f96956bbd1c6bf0c623f249265915025eb7"} Jul 09 14:06:06.121629 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:06.121604 2575 generic.go:356] "Generic (PLEG): container finished" podID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerID="bcd7e43780d5c47a4b3bbaeca369709bd4a4b50dafb99ac27490baacfca746d8" exitCode=0 Jul 09 14:06:06.121741 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:06.121688 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" event={"ID":"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1","Type":"ContainerDied","Data":"bcd7e43780d5c47a4b3bbaeca369709bd4a4b50dafb99ac27490baacfca746d8"} Jul 09 14:06:06.121741 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:06.121729 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" event={"ID":"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1","Type":"ContainerStarted","Data":"092b8e2140ff9cd3d28ff2a79e4954ddb46159ab51b60e52017a22ee8c7a2287"} Jul 09 14:06:06.123906 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:06.123877 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerID="174275929ac95c4b162ba4981471a7cfe3996837752b197217d53d68cb267405" exitCode=0 Jul 09 14:06:06.124005 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:06.123941 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" event={"ID":"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4","Type":"ContainerDied","Data":"174275929ac95c4b162ba4981471a7cfe3996837752b197217d53d68cb267405"} Jul 09 14:06:07.132242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.132205 2575 generic.go:356] "Generic (PLEG): container finished" podID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerID="c1345092951be333712e7d9bbd88f0f467a6556156d157e697c80acd28af6e6f" exitCode=0 Jul 09 14:06:07.132682 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.132287 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" event={"ID":"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1","Type":"ContainerDied","Data":"c1345092951be333712e7d9bbd88f0f467a6556156d157e697c80acd28af6e6f"} Jul 09 14:06:07.388710 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.388644 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:07.392029 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.392003 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:07.473289 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.473260 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k27rk\" (UniqueName: \"kubernetes.io/projected/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-kube-api-access-k27rk\") pod \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " Jul 09 14:06:07.473289 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.473292 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj7md\" (UniqueName: \"kubernetes.io/projected/aaeb587b-9833-486c-a551-a8a1ee38a561-kube-api-access-lj7md\") pod \"aaeb587b-9833-486c-a551-a8a1ee38a561\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " Jul 09 14:06:07.474026 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.473314 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-util\") pod \"aaeb587b-9833-486c-a551-a8a1ee38a561\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " Jul 09 14:06:07.474026 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.473342 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-util\") pod \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " Jul 09 14:06:07.474026 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.473390 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-bundle\") pod \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\" (UID: \"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4\") " Jul 09 14:06:07.474026 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.473422 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-bundle\") pod \"aaeb587b-9833-486c-a551-a8a1ee38a561\" (UID: \"aaeb587b-9833-486c-a551-a8a1ee38a561\") " Jul 09 14:06:07.474026 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.473971 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-bundle" (OuterVolumeSpecName: "bundle") pod "ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" (UID: "ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:07.474443 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.474332 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-bundle" (OuterVolumeSpecName: "bundle") pod "aaeb587b-9833-486c-a551-a8a1ee38a561" (UID: "aaeb587b-9833-486c-a551-a8a1ee38a561"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:07.475938 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.475888 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-kube-api-access-k27rk" (OuterVolumeSpecName: "kube-api-access-k27rk") pod "ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" (UID: "ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4"). InnerVolumeSpecName "kube-api-access-k27rk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:06:07.476054 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.475934 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaeb587b-9833-486c-a551-a8a1ee38a561-kube-api-access-lj7md" (OuterVolumeSpecName: "kube-api-access-lj7md") pod "aaeb587b-9833-486c-a551-a8a1ee38a561" (UID: "aaeb587b-9833-486c-a551-a8a1ee38a561"). InnerVolumeSpecName "kube-api-access-lj7md". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:06:07.478963 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.478942 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-util" (OuterVolumeSpecName: "util") pod "ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" (UID: "ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:07.479015 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.478961 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-util" (OuterVolumeSpecName: "util") pod "aaeb587b-9833-486c-a551-a8a1ee38a561" (UID: "aaeb587b-9833-486c-a551-a8a1ee38a561"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:07.574242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.574208 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k27rk\" (UniqueName: \"kubernetes.io/projected/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-kube-api-access-k27rk\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:07.574242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.574238 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj7md\" (UniqueName: \"kubernetes.io/projected/aaeb587b-9833-486c-a551-a8a1ee38a561-kube-api-access-lj7md\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:07.574242 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.574248 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-util\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:07.574464 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.574258 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-util\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:07.574464 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.574266 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4-bundle\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:07.574464 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:07.574273 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaeb587b-9833-486c-a551-a8a1ee38a561-bundle\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:08.139815 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.139786 2575 generic.go:356] "Generic (PLEG): container finished" podID="4152889b-5503-4d27-a0ec-33d4722e2652" containerID="a1acb111378c143aed364813c8d4eccaa91e0917c69a7eb40dce879fbb8f6b54" exitCode=0 Jul 09 14:06:08.140297 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.139887 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" event={"ID":"4152889b-5503-4d27-a0ec-33d4722e2652","Type":"ContainerDied","Data":"a1acb111378c143aed364813c8d4eccaa91e0917c69a7eb40dce879fbb8f6b54"} Jul 09 14:06:08.142060 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.142041 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" Jul 09 14:06:08.142060 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.142047 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774" event={"ID":"ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4","Type":"ContainerDied","Data":"bd333b9fffc84d8d3242cc32b9bf25bee241b2e78a24dfd7b4489af0bbfa66ef"} Jul 09 14:06:08.142245 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.142090 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd333b9fffc84d8d3242cc32b9bf25bee241b2e78a24dfd7b4489af0bbfa66ef" Jul 09 14:06:08.143987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.143958 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" Jul 09 14:06:08.143987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.143976 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx" event={"ID":"aaeb587b-9833-486c-a551-a8a1ee38a561","Type":"ContainerDied","Data":"eef12f65fb10e16ab71867d5c2fadad221347882ec90016d0e8968eab7dcb4a6"} Jul 09 14:06:08.144124 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.144020 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eef12f65fb10e16ab71867d5c2fadad221347882ec90016d0e8968eab7dcb4a6" Jul 09 14:06:08.146535 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.146515 2575 generic.go:356] "Generic (PLEG): container finished" podID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerID="62608543c235f01b4e84cb57055c62cc7f5fc1659986b377c450eb2d88e75f94" exitCode=0 Jul 09 14:06:08.146668 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:08.146568 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" event={"ID":"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1","Type":"ContainerDied","Data":"62608543c235f01b4e84cb57055c62cc7f5fc1659986b377c450eb2d88e75f94"} Jul 09 14:06:09.154603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.154565 2575 generic.go:356] "Generic (PLEG): container finished" podID="4152889b-5503-4d27-a0ec-33d4722e2652" containerID="877f1d94ffa32c918be4a459ebfa691913b3ce0e5e94445a8c202ec24b2fe589" exitCode=0 Jul 09 14:06:09.155007 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.154649 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" event={"ID":"4152889b-5503-4d27-a0ec-33d4722e2652","Type":"ContainerDied","Data":"877f1d94ffa32c918be4a459ebfa691913b3ce0e5e94445a8c202ec24b2fe589"} Jul 09 14:06:09.298922 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.298896 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:09.390525 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.390485 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-util\") pod \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " Jul 09 14:06:09.390711 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.390572 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-bundle\") pod \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " Jul 09 14:06:09.390711 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.390592 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsqb4\" (UniqueName: \"kubernetes.io/projected/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-kube-api-access-xsqb4\") pod \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\" (UID: \"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1\") " Jul 09 14:06:09.391117 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.391087 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-bundle" (OuterVolumeSpecName: "bundle") pod "fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" (UID: "fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:09.392955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.392925 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-kube-api-access-xsqb4" (OuterVolumeSpecName: "kube-api-access-xsqb4") pod "fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" (UID: "fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1"). InnerVolumeSpecName "kube-api-access-xsqb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:06:09.396260 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.396234 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-util" (OuterVolumeSpecName: "util") pod "fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" (UID: "fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:09.491762 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.491675 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-bundle\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:09.491762 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.491708 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsqb4\" (UniqueName: \"kubernetes.io/projected/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-kube-api-access-xsqb4\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:09.491762 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:09.491718 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1-util\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:10.163247 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.163221 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" Jul 09 14:06:10.163622 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.163263 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8" event={"ID":"fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1","Type":"ContainerDied","Data":"092b8e2140ff9cd3d28ff2a79e4954ddb46159ab51b60e52017a22ee8c7a2287"} Jul 09 14:06:10.163622 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.163311 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="092b8e2140ff9cd3d28ff2a79e4954ddb46159ab51b60e52017a22ee8c7a2287" Jul 09 14:06:10.302955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.302923 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:10.400532 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.400493 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-bundle\") pod \"4152889b-5503-4d27-a0ec-33d4722e2652\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " Jul 09 14:06:10.400727 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.400596 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klc47\" (UniqueName: \"kubernetes.io/projected/4152889b-5503-4d27-a0ec-33d4722e2652-kube-api-access-klc47\") pod \"4152889b-5503-4d27-a0ec-33d4722e2652\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " Jul 09 14:06:10.400727 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.400651 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-util\") pod \"4152889b-5503-4d27-a0ec-33d4722e2652\" (UID: \"4152889b-5503-4d27-a0ec-33d4722e2652\") " Jul 09 14:06:10.401047 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.401017 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-bundle" (OuterVolumeSpecName: "bundle") pod "4152889b-5503-4d27-a0ec-33d4722e2652" (UID: "4152889b-5503-4d27-a0ec-33d4722e2652"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:10.402912 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.402870 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4152889b-5503-4d27-a0ec-33d4722e2652-kube-api-access-klc47" (OuterVolumeSpecName: "kube-api-access-klc47") pod "4152889b-5503-4d27-a0ec-33d4722e2652" (UID: "4152889b-5503-4d27-a0ec-33d4722e2652"). InnerVolumeSpecName "kube-api-access-klc47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:06:10.405098 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.405076 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-util" (OuterVolumeSpecName: "util") pod "4152889b-5503-4d27-a0ec-33d4722e2652" (UID: "4152889b-5503-4d27-a0ec-33d4722e2652"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:06:10.501446 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.501402 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-util\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:10.501446 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.501437 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4152889b-5503-4d27-a0ec-33d4722e2652-bundle\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:10.501446 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:10.501451 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klc47\" (UniqueName: \"kubernetes.io/projected/4152889b-5503-4d27-a0ec-33d4722e2652-kube-api-access-klc47\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:11.170193 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:11.170162 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" Jul 09 14:06:11.170193 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:11.170189 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr" event={"ID":"4152889b-5503-4d27-a0ec-33d4722e2652","Type":"ContainerDied","Data":"0a938623a02b53c4339b3587ad89afcab68bc7e4f5bb8e0b6b3bd36c060082ff"} Jul 09 14:06:11.170649 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:11.170238 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a938623a02b53c4339b3587ad89afcab68bc7e4f5bb8e0b6b3bd36c060082ff" Jul 09 14:06:21.037477 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:21.037437 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7677c9696b-8fwdk"] Jul 09 14:06:27.888959 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.888926 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq"] Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889309 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerName="pull" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889321 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerName="pull" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889334 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4152889b-5503-4d27-a0ec-33d4722e2652" containerName="extract" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889344 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4152889b-5503-4d27-a0ec-33d4722e2652" containerName="extract" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889354 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerName="util" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889360 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerName="util" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889367 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4152889b-5503-4d27-a0ec-33d4722e2652" containerName="util" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889372 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4152889b-5503-4d27-a0ec-33d4722e2652" containerName="util" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889378 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerName="pull" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889383 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerName="pull" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889391 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerName="pull" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889397 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerName="pull" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889403 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerName="extract" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889407 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerName="extract" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889415 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerName="util" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889423 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerName="util" Jul 09 14:06:27.889427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889436 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4152889b-5503-4d27-a0ec-33d4722e2652" containerName="pull" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889441 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4152889b-5503-4d27-a0ec-33d4722e2652" containerName="pull" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889448 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerName="extract" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889453 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerName="extract" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889464 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerName="util" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889468 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerName="util" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889474 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerName="extract" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889479 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerName="extract" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889542 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1" containerName="extract" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889550 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaeb587b-9833-486c-a551-a8a1ee38a561" containerName="extract" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889556 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4" containerName="extract" Jul 09 14:06:27.889974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.889562 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4152889b-5503-4d27-a0ec-33d4722e2652" containerName="extract" Jul 09 14:06:27.893736 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.893718 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:27.896351 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.896327 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 14:06:27.896494 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.896442 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 14:06:27.897393 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.897376 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bw2bd\"" Jul 09 14:06:27.902722 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:27.902698 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq"] Jul 09 14:06:28.070661 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:28.070616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq8rt\" (UniqueName: \"kubernetes.io/projected/67b20dcb-2415-4f6a-9617-786fdb187a84-kube-api-access-rq8rt\") pod \"limitador-operator-controller-manager-859d7b78bd-qf7nq\" (UID: \"67b20dcb-2415-4f6a-9617-786fdb187a84\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:28.172054 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:28.171952 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq8rt\" (UniqueName: \"kubernetes.io/projected/67b20dcb-2415-4f6a-9617-786fdb187a84-kube-api-access-rq8rt\") pod \"limitador-operator-controller-manager-859d7b78bd-qf7nq\" (UID: \"67b20dcb-2415-4f6a-9617-786fdb187a84\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:28.183747 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:28.183719 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq8rt\" (UniqueName: \"kubernetes.io/projected/67b20dcb-2415-4f6a-9617-786fdb187a84-kube-api-access-rq8rt\") pod \"limitador-operator-controller-manager-859d7b78bd-qf7nq\" (UID: \"67b20dcb-2415-4f6a-9617-786fdb187a84\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:28.205343 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:28.205305 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:28.384052 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:28.384023 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq"] Jul 09 14:06:28.386466 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:06:28.386437 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b20dcb_2415_4f6a_9617_786fdb187a84.slice/crio-a49322a2a85f50a4a775a0881c17ee5eeef7b73bf953e013ea0bf08acf4841f3 WatchSource:0}: Error finding container a49322a2a85f50a4a775a0881c17ee5eeef7b73bf953e013ea0bf08acf4841f3: Status 404 returned error can't find the container with id a49322a2a85f50a4a775a0881c17ee5eeef7b73bf953e013ea0bf08acf4841f3 Jul 09 14:06:29.288474 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:29.288429 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" event={"ID":"67b20dcb-2415-4f6a-9617-786fdb187a84","Type":"ContainerStarted","Data":"a49322a2a85f50a4a775a0881c17ee5eeef7b73bf953e013ea0bf08acf4841f3"} Jul 09 14:06:31.303371 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:31.303332 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" event={"ID":"67b20dcb-2415-4f6a-9617-786fdb187a84","Type":"ContainerStarted","Data":"03282ca4692dbc17747ab3dcfb14d1b9b774d08e7c76d71a7f9701809c778599"} Jul 09 14:06:31.303840 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:31.303433 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:31.326426 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:31.326362 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" podStartSLOduration=2.4820117870000002 podStartE2EDuration="4.326345505s" totalImagesPullingTime="1.844333718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:06:31.325237704 +0000 UTC m=+655.179189181" watchObservedRunningTime="2026-07-09 14:06:31.326345505 +0000 UTC m=+655.180296993" Jul 09 14:06:32.940445 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:32.940404 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv"] Jul 09 14:06:32.944249 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:32.944225 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:32.947524 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:32.947502 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 14:06:32.947658 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:32.947589 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vh7wf\"" Jul 09 14:06:32.948252 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:32.948235 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 14:06:32.958688 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:32.958659 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv"] Jul 09 14:06:33.115677 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.115631 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhwld\" (UniqueName: \"kubernetes.io/projected/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-kube-api-access-lhwld\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.115889 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.115712 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.115889 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.115737 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.217225 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.217126 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.217225 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.217182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.217473 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.217272 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhwld\" (UniqueName: \"kubernetes.io/projected/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-kube-api-access-lhwld\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.217473 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:33.217284 2575 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 09 14:06:33.217473 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:33.217400 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-plugin-serving-cert podName:ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2 nodeName:}" failed. No retries permitted until 2026-07-09 14:06:33.717378097 +0000 UTC m=+657.571329567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-kd7cv" (UID: "ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2") : secret "plugin-serving-cert" not found Jul 09 14:06:33.217956 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.217933 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.226778 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.226751 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhwld\" (UniqueName: \"kubernetes.io/projected/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-kube-api-access-lhwld\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.722228 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.722186 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.724827 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.724805 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kd7cv\" (UID: \"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.854115 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.854074 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" Jul 09 14:06:33.995000 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:33.994977 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv"] Jul 09 14:06:33.996902 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:06:33.996876 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba1cdfc3_34e6_4dac_80e1_6aa58fe7d8b2.slice/crio-6584936a42099da28598ce0782ff4d8483758cdc425ce150e079d03c41665009 WatchSource:0}: Error finding container 6584936a42099da28598ce0782ff4d8483758cdc425ce150e079d03c41665009: Status 404 returned error can't find the container with id 6584936a42099da28598ce0782ff4d8483758cdc425ce150e079d03c41665009 Jul 09 14:06:34.320748 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:34.320718 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" event={"ID":"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2","Type":"ContainerStarted","Data":"6584936a42099da28598ce0782ff4d8483758cdc425ce150e079d03c41665009"} Jul 09 14:06:42.312867 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:42.312817 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:43.845588 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.845547 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq"] Jul 09 14:06:43.846047 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.845793 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" containerName="manager" containerID="cri-o://03282ca4692dbc17747ab3dcfb14d1b9b774d08e7c76d71a7f9701809c778599" gracePeriod=2 Jul 09 14:06:43.849423 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.849395 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq"] Jul 09 14:06:43.852727 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:43.852674 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qf7nq\" is forbidden: User \"system:node:ip-10-0-128-190.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-190.ec2.internal' and this object" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:43.883285 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.883249 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg"] Jul 09 14:06:43.883871 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.883822 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" containerName="manager" Jul 09 14:06:43.883871 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.883863 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" containerName="manager" Jul 09 14:06:43.884293 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.883962 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" containerName="manager" Jul 09 14:06:43.887927 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.887904 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" Jul 09 14:06:43.890397 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:43.890357 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qf7nq\" is forbidden: User \"system:node:ip-10-0-128-190.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-190.ec2.internal' and this object" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:43.899579 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.899486 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg"] Jul 09 14:06:43.919013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:43.918976 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z47lb\" (UniqueName: \"kubernetes.io/projected/4b1344c0-fe58-4542-98a3-330a040be406-kube-api-access-z47lb\") pod \"limitador-operator-controller-manager-859d7b78bd-xl7qg\" (UID: \"4b1344c0-fe58-4542-98a3-330a040be406\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" Jul 09 14:06:44.020606 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:44.020561 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z47lb\" (UniqueName: \"kubernetes.io/projected/4b1344c0-fe58-4542-98a3-330a040be406-kube-api-access-z47lb\") pod \"limitador-operator-controller-manager-859d7b78bd-xl7qg\" (UID: \"4b1344c0-fe58-4542-98a3-330a040be406\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" Jul 09 14:06:44.037158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:44.037125 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z47lb\" (UniqueName: \"kubernetes.io/projected/4b1344c0-fe58-4542-98a3-330a040be406-kube-api-access-z47lb\") pod \"limitador-operator-controller-manager-859d7b78bd-xl7qg\" (UID: \"4b1344c0-fe58-4542-98a3-330a040be406\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" Jul 09 14:06:44.219013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:44.218910 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" Jul 09 14:06:46.067417 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:46.067377 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7677c9696b-8fwdk" podUID="b950e6be-8973-4183-bd1a-1359bbb7597f" containerName="console" containerID="cri-o://de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada" gracePeriod=15 Jul 09 14:06:55.998192 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:55.998137 2575 patch_prober.go:28] interesting pod/console-7677c9696b-8fwdk container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.23:8443/health\": dial tcp 10.132.0.23:8443: connect: connection refused" start-of-body= Jul 09 14:06:55.998752 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:55.998225 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-7677c9696b-8fwdk" podUID="b950e6be-8973-4183-bd1a-1359bbb7597f" containerName="console" probeResult="failure" output="Get \"https://10.132.0.23:8443/health\": dial tcp 10.132.0.23:8443: connect: connection refused" Jul 09 14:06:58.404573 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.404542 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg"] Jul 09 14:06:58.445198 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.445158 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:06:58.487494 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.487390 2575 generic.go:356] "Generic (PLEG): container finished" podID="b950e6be-8973-4183-bd1a-1359bbb7597f" containerID="de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada" exitCode=0 Jul 09 14:06:58.487494 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.487428 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7677c9696b-8fwdk" Jul 09 14:06:58.487494 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.487465 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677c9696b-8fwdk" event={"ID":"b950e6be-8973-4183-bd1a-1359bbb7597f","Type":"ContainerDied","Data":"de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada"} Jul 09 14:06:58.487844 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.487516 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7677c9696b-8fwdk" event={"ID":"b950e6be-8973-4183-bd1a-1359bbb7597f","Type":"ContainerDied","Data":"48d21d07a4c8ef0cd836e8954058983f2ab31d3aee411a7cbd749dc9f994d26e"} Jul 09 14:06:58.487844 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.487542 2575 scope.go:117] "RemoveContainer" containerID="de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada" Jul 09 14:06:58.489534 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.489508 2575 generic.go:356] "Generic (PLEG): container finished" podID="67b20dcb-2415-4f6a-9617-786fdb187a84" containerID="03282ca4692dbc17747ab3dcfb14d1b9b774d08e7c76d71a7f9701809c778599" exitCode=0 Jul 09 14:06:58.566105 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566070 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-trusted-ca-bundle\") pod \"b950e6be-8973-4183-bd1a-1359bbb7597f\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " Jul 09 14:06:58.566105 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566120 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-oauth-config\") pod \"b950e6be-8973-4183-bd1a-1359bbb7597f\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " Jul 09 14:06:58.566349 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566175 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-oauth-serving-cert\") pod \"b950e6be-8973-4183-bd1a-1359bbb7597f\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " Jul 09 14:06:58.566349 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566196 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-serving-cert\") pod \"b950e6be-8973-4183-bd1a-1359bbb7597f\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " Jul 09 14:06:58.566349 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566219 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47px5\" (UniqueName: \"kubernetes.io/projected/b950e6be-8973-4183-bd1a-1359bbb7597f-kube-api-access-47px5\") pod \"b950e6be-8973-4183-bd1a-1359bbb7597f\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " Jul 09 14:06:58.566349 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566254 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-console-config\") pod \"b950e6be-8973-4183-bd1a-1359bbb7597f\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " Jul 09 14:06:58.566349 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566335 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-service-ca\") pod \"b950e6be-8973-4183-bd1a-1359bbb7597f\" (UID: \"b950e6be-8973-4183-bd1a-1359bbb7597f\") " Jul 09 14:06:58.566581 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566519 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b950e6be-8973-4183-bd1a-1359bbb7597f" (UID: "b950e6be-8973-4183-bd1a-1359bbb7597f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:06:58.566668 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566610 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b950e6be-8973-4183-bd1a-1359bbb7597f" (UID: "b950e6be-8973-4183-bd1a-1359bbb7597f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:06:58.566809 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566787 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-trusted-ca-bundle\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:58.566809 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566796 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-console-config" (OuterVolumeSpecName: "console-config") pod "b950e6be-8973-4183-bd1a-1359bbb7597f" (UID: "b950e6be-8973-4183-bd1a-1359bbb7597f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:06:58.566992 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.566935 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-service-ca" (OuterVolumeSpecName: "service-ca") pod "b950e6be-8973-4183-bd1a-1359bbb7597f" (UID: "b950e6be-8973-4183-bd1a-1359bbb7597f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:06:58.568623 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.568602 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b950e6be-8973-4183-bd1a-1359bbb7597f" (UID: "b950e6be-8973-4183-bd1a-1359bbb7597f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:06:58.568719 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.568633 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b950e6be-8973-4183-bd1a-1359bbb7597f-kube-api-access-47px5" (OuterVolumeSpecName: "kube-api-access-47px5") pod "b950e6be-8973-4183-bd1a-1359bbb7597f" (UID: "b950e6be-8973-4183-bd1a-1359bbb7597f"). InnerVolumeSpecName "kube-api-access-47px5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:06:58.568719 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.568656 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b950e6be-8973-4183-bd1a-1359bbb7597f" (UID: "b950e6be-8973-4183-bd1a-1359bbb7597f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:06:58.597862 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:06:58.597815 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b1344c0_fe58_4542_98a3_330a040be406.slice/crio-d340d197de6663573e8b5e6e4624e56df199c993604d2355375397fb628735fe WatchSource:0}: Error finding container d340d197de6663573e8b5e6e4624e56df199c993604d2355375397fb628735fe: Status 404 returned error can't find the container with id d340d197de6663573e8b5e6e4624e56df199c993604d2355375397fb628735fe Jul 09 14:06:58.630905 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.630842 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:58.633371 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:58.633330 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qf7nq\" is forbidden: User \"system:node:ip-10-0-128-190.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-190.ec2.internal' and this object" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:58.646385 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.646091 2575 scope.go:117] "RemoveContainer" containerID="de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada" Jul 09 14:06:58.646546 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:58.646521 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada\": container with ID starting with de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada not found: ID does not exist" containerID="de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada" Jul 09 14:06:58.646604 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.646563 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada"} err="failed to get container status \"de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada\": rpc error: code = NotFound desc = could not find container \"de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada\": container with ID starting with de7cec6971aa5e3f6d0e201977227a73b9ae0c7d6c9b0446a334bfae0a299ada not found: ID does not exist" Jul 09 14:06:58.667705 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.667669 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-service-ca\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:58.667705 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.667699 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-oauth-config\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:58.667705 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.667711 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-oauth-serving-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:58.667951 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.667720 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b950e6be-8973-4183-bd1a-1359bbb7597f-console-serving-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:58.667951 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.667729 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47px5\" (UniqueName: \"kubernetes.io/projected/b950e6be-8973-4183-bd1a-1359bbb7597f-kube-api-access-47px5\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:58.667951 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.667738 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b950e6be-8973-4183-bd1a-1359bbb7597f-console-config\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:58.768732 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.768709 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq8rt\" (UniqueName: \"kubernetes.io/projected/67b20dcb-2415-4f6a-9617-786fdb187a84-kube-api-access-rq8rt\") pod \"67b20dcb-2415-4f6a-9617-786fdb187a84\" (UID: \"67b20dcb-2415-4f6a-9617-786fdb187a84\") " Jul 09 14:06:58.770912 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.770877 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67b20dcb-2415-4f6a-9617-786fdb187a84-kube-api-access-rq8rt" (OuterVolumeSpecName: "kube-api-access-rq8rt") pod "67b20dcb-2415-4f6a-9617-786fdb187a84" (UID: "67b20dcb-2415-4f6a-9617-786fdb187a84"). InnerVolumeSpecName "kube-api-access-rq8rt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:06:58.813910 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.813839 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7677c9696b-8fwdk"] Jul 09 14:06:58.816546 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.816518 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7677c9696b-8fwdk"] Jul 09 14:06:58.870225 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:58.870187 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq8rt\" (UniqueName: \"kubernetes.io/projected/67b20dcb-2415-4f6a-9617-786fdb187a84-kube-api-access-rq8rt\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:06:59.500197 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:59.500111 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:59.500197 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:59.500157 2575 scope.go:117] "RemoveContainer" containerID="03282ca4692dbc17747ab3dcfb14d1b9b774d08e7c76d71a7f9701809c778599" Jul 09 14:06:59.501813 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:59.501777 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" event={"ID":"ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2","Type":"ContainerStarted","Data":"3d2cade194610b47e02b7492203c69edc4e4ff3713e08fefca4af95d43929194"} Jul 09 14:06:59.503144 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:59.503119 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qf7nq\" is forbidden: User \"system:node:ip-10-0-128-190.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-190.ec2.internal' and this object" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:59.503562 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:59.503541 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" event={"ID":"4b1344c0-fe58-4542-98a3-330a040be406","Type":"ContainerStarted","Data":"4d3ea025ecc7a649a3bdaa8aefd76b4a80091ede1ede25b6acb484fc1f267bdf"} Jul 09 14:06:59.503659 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:59.503590 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" event={"ID":"4b1344c0-fe58-4542-98a3-330a040be406","Type":"ContainerStarted","Data":"d340d197de6663573e8b5e6e4624e56df199c993604d2355375397fb628735fe"} Jul 09 14:06:59.503659 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:59.503620 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" Jul 09 14:06:59.505486 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:59.505457 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qf7nq\" is forbidden: User \"system:node:ip-10-0-128-190.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-190.ec2.internal' and this object" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:59.548031 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:59.547953 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kd7cv" podStartSLOduration=2.863312886 podStartE2EDuration="27.547936177s" totalImagesPullingTime="24.684623291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:06:59.5456397 +0000 UTC m=+683.399591176" watchObservedRunningTime="2026-07-09 14:06:59.547936177 +0000 UTC m=+683.401887643" Jul 09 14:06:59.548258 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:06:59.548225 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-qf7nq\" is forbidden: User \"system:node:ip-10-0-128-190.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-190.ec2.internal' and this object" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qf7nq" Jul 09 14:06:59.588343 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:06:59.588288 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" podStartSLOduration=16.58826897 podStartE2EDuration="16.58826897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:06:59.585250837 +0000 UTC m=+683.439202313" watchObservedRunningTime="2026-07-09 14:06:59.58826897 +0000 UTC m=+683.442220458" Jul 09 14:07:00.637466 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:00.637415 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67b20dcb-2415-4f6a-9617-786fdb187a84" path="/var/lib/kubelet/pods/67b20dcb-2415-4f6a-9617-786fdb187a84/volumes" Jul 09 14:07:00.638027 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:00.638007 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b950e6be-8973-4183-bd1a-1359bbb7597f" path="/var/lib/kubelet/pods/b950e6be-8973-4183-bd1a-1359bbb7597f/volumes" Jul 09 14:07:10.519018 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:10.518985 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xl7qg" Jul 09 14:07:14.483625 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.483591 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs"] Jul 09 14:07:14.484043 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.484017 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b950e6be-8973-4183-bd1a-1359bbb7597f" containerName="console" Jul 09 14:07:14.484043 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.484030 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b950e6be-8973-4183-bd1a-1359bbb7597f" containerName="console" Jul 09 14:07:14.484118 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.484094 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b950e6be-8973-4183-bd1a-1359bbb7597f" containerName="console" Jul 09 14:07:14.507911 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.507876 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs"] Jul 09 14:07:14.508078 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.507997 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.510732 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.510709 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xtk6l\"" Jul 09 14:07:14.608363 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.608322 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.608559 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.608373 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h4vs\" (UniqueName: \"kubernetes.io/projected/08e20f32-3d40-470c-b35f-7b288fb5bb66-kube-api-access-8h4vs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.608559 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.608481 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.608559 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.608525 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.608559 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.608543 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.608729 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.608566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.608729 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.608584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08e20f32-3d40-470c-b35f-7b288fb5bb66-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.608729 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.608603 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.608729 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.608627 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710064 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710025 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710064 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710072 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710091 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710115 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710130 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08e20f32-3d40-470c-b35f-7b288fb5bb66-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710149 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710253 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710278 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h4vs\" (UniqueName: \"kubernetes.io/projected/08e20f32-3d40-470c-b35f-7b288fb5bb66-kube-api-access-8h4vs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710720 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710564 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710720 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710706 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710810 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710784 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.710991 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.710971 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08e20f32-3d40-470c-b35f-7b288fb5bb66-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.711184 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.711164 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.712755 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.712738 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.712990 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.712971 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.718958 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.718929 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08e20f32-3d40-470c-b35f-7b288fb5bb66-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.718958 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.718946 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h4vs\" (UniqueName: \"kubernetes.io/projected/08e20f32-3d40-470c-b35f-7b288fb5bb66-kube-api-access-8h4vs\") pod \"maas-default-gateway-openshift-default-56654dcf7-2njfs\" (UID: \"08e20f32-3d40-470c-b35f-7b288fb5bb66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.821533 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.821499 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:14.967036 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.967003 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs"] Jul 09 14:07:14.967755 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:07:14.967725 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e20f32_3d40_470c_b35f_7b288fb5bb66.slice/crio-3a9f249da751e39c52d23692f1e3488547f57f17c72bde62e36e9ab4111164b1 WatchSource:0}: Error finding container 3a9f249da751e39c52d23692f1e3488547f57f17c72bde62e36e9ab4111164b1: Status 404 returned error can't find the container with id 3a9f249da751e39c52d23692f1e3488547f57f17c72bde62e36e9ab4111164b1 Jul 09 14:07:14.970107 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.970071 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:07:14.970218 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.970155 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:07:14.970218 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:14.970202 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:07:15.604980 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:15.604943 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" event={"ID":"08e20f32-3d40-470c-b35f-7b288fb5bb66","Type":"ContainerStarted","Data":"da9b3c8973bfbc57a4e689ea353d9384f802059adf21908722e75bb3a9bc0ba3"} Jul 09 14:07:15.605494 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:15.605007 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" event={"ID":"08e20f32-3d40-470c-b35f-7b288fb5bb66","Type":"ContainerStarted","Data":"3a9f249da751e39c52d23692f1e3488547f57f17c72bde62e36e9ab4111164b1"} Jul 09 14:07:15.627133 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:15.627076 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" podStartSLOduration=1.627061176 podStartE2EDuration="1.627061176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:15.624330981 +0000 UTC m=+699.478282457" watchObservedRunningTime="2026-07-09 14:07:15.627061176 +0000 UTC m=+699.481012718" Jul 09 14:07:15.821782 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:15.821737 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:16.827020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:16.826985 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:17.617270 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:17.617230 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:17.618303 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:17.618286 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:07:17.771807 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:17.771763 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-n7wkt"] Jul 09 14:07:17.774571 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:17.774547 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n7wkt" Jul 09 14:07:17.777083 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:17.777062 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-85j9k\"" Jul 09 14:07:17.781153 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:17.781125 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n7wkt"] Jul 09 14:07:17.941830 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:17.941724 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b87hj\" (UniqueName: \"kubernetes.io/projected/3aa0c6c0-3b7b-4748-b5ec-067e5115eb57-kube-api-access-b87hj\") pod \"authorino-75d556bd77-n7wkt\" (UID: \"3aa0c6c0-3b7b-4748-b5ec-067e5115eb57\") " pod="kuadrant-system/authorino-75d556bd77-n7wkt" Jul 09 14:07:18.042664 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:18.042620 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b87hj\" (UniqueName: \"kubernetes.io/projected/3aa0c6c0-3b7b-4748-b5ec-067e5115eb57-kube-api-access-b87hj\") pod \"authorino-75d556bd77-n7wkt\" (UID: \"3aa0c6c0-3b7b-4748-b5ec-067e5115eb57\") " pod="kuadrant-system/authorino-75d556bd77-n7wkt" Jul 09 14:07:18.050951 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:18.050919 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b87hj\" (UniqueName: \"kubernetes.io/projected/3aa0c6c0-3b7b-4748-b5ec-067e5115eb57-kube-api-access-b87hj\") pod \"authorino-75d556bd77-n7wkt\" (UID: \"3aa0c6c0-3b7b-4748-b5ec-067e5115eb57\") " pod="kuadrant-system/authorino-75d556bd77-n7wkt" Jul 09 14:07:18.085746 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:18.085717 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n7wkt" Jul 09 14:07:18.223331 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:18.223305 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n7wkt"] Jul 09 14:07:18.225309 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:07:18.225277 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3aa0c6c0_3b7b_4748_b5ec_067e5115eb57.slice/crio-d34ea00083c1f0ca5bec989981d32ffa6b653988a8cb2bbf3ec29790f6731ed1 WatchSource:0}: Error finding container d34ea00083c1f0ca5bec989981d32ffa6b653988a8cb2bbf3ec29790f6731ed1: Status 404 returned error can't find the container with id d34ea00083c1f0ca5bec989981d32ffa6b653988a8cb2bbf3ec29790f6731ed1 Jul 09 14:07:18.623770 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:18.623733 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n7wkt" event={"ID":"3aa0c6c0-3b7b-4748-b5ec-067e5115eb57","Type":"ContainerStarted","Data":"d34ea00083c1f0ca5bec989981d32ffa6b653988a8cb2bbf3ec29790f6731ed1"} Jul 09 14:07:22.651566 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:22.651526 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n7wkt" event={"ID":"3aa0c6c0-3b7b-4748-b5ec-067e5115eb57","Type":"ContainerStarted","Data":"49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226"} Jul 09 14:07:22.671618 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:22.671561 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-n7wkt" podStartSLOduration=1.386044102 podStartE2EDuration="5.671544227s" totalImagesPullingTime="4.285500125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:22.668719762 +0000 UTC m=+706.522671239" watchObservedRunningTime="2026-07-09 14:07:22.671544227 +0000 UTC m=+706.525495702" Jul 09 14:07:52.431141 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.431106 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n7wkt"] Jul 09 14:07:52.431599 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.431299 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-n7wkt" podUID="3aa0c6c0-3b7b-4748-b5ec-067e5115eb57" containerName="authorino" containerID="cri-o://49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226" gracePeriod=30 Jul 09 14:07:52.685866 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.685774 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n7wkt" Jul 09 14:07:52.824587 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.824554 2575 generic.go:356] "Generic (PLEG): container finished" podID="3aa0c6c0-3b7b-4748-b5ec-067e5115eb57" containerID="49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226" exitCode=0 Jul 09 14:07:52.824760 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.824599 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-n7wkt" Jul 09 14:07:52.824760 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.824643 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n7wkt" event={"ID":"3aa0c6c0-3b7b-4748-b5ec-067e5115eb57","Type":"ContainerDied","Data":"49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226"} Jul 09 14:07:52.824760 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.824689 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-n7wkt" event={"ID":"3aa0c6c0-3b7b-4748-b5ec-067e5115eb57","Type":"ContainerDied","Data":"d34ea00083c1f0ca5bec989981d32ffa6b653988a8cb2bbf3ec29790f6731ed1"} Jul 09 14:07:52.824760 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.824710 2575 scope.go:117] "RemoveContainer" containerID="49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226" Jul 09 14:07:52.834329 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.834309 2575 scope.go:117] "RemoveContainer" containerID="49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226" Jul 09 14:07:52.834631 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:07:52.834610 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226\": container with ID starting with 49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226 not found: ID does not exist" containerID="49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226" Jul 09 14:07:52.834712 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.834646 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226"} err="failed to get container status \"49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226\": rpc error: code = NotFound desc = could not find container \"49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226\": container with ID starting with 49862f0b47b9dfb2b1db9fc733bca1d38b533c1811024829b36d428727aa9226 not found: ID does not exist" Jul 09 14:07:52.861060 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.861026 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b87hj\" (UniqueName: \"kubernetes.io/projected/3aa0c6c0-3b7b-4748-b5ec-067e5115eb57-kube-api-access-b87hj\") pod \"3aa0c6c0-3b7b-4748-b5ec-067e5115eb57\" (UID: \"3aa0c6c0-3b7b-4748-b5ec-067e5115eb57\") " Jul 09 14:07:52.863274 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.863248 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aa0c6c0-3b7b-4748-b5ec-067e5115eb57-kube-api-access-b87hj" (OuterVolumeSpecName: "kube-api-access-b87hj") pod "3aa0c6c0-3b7b-4748-b5ec-067e5115eb57" (UID: "3aa0c6c0-3b7b-4748-b5ec-067e5115eb57"). InnerVolumeSpecName "kube-api-access-b87hj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:07:52.962214 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:52.962126 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b87hj\" (UniqueName: \"kubernetes.io/projected/3aa0c6c0-3b7b-4748-b5ec-067e5115eb57-kube-api-access-b87hj\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:07:53.150704 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:53.150667 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n7wkt"] Jul 09 14:07:53.163399 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:53.163354 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-n7wkt"] Jul 09 14:07:54.637905 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:54.637870 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aa0c6c0-3b7b-4748-b5ec-067e5115eb57" path="/var/lib/kubelet/pods/3aa0c6c0-3b7b-4748-b5ec-067e5115eb57/volumes" Jul 09 14:07:56.694886 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.694832 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d4777695c-mmfhr"] Jul 09 14:07:56.695417 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.695401 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa0c6c0-3b7b-4748-b5ec-067e5115eb57" containerName="authorino" Jul 09 14:07:56.695471 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.695421 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa0c6c0-3b7b-4748-b5ec-067e5115eb57" containerName="authorino" Jul 09 14:07:56.695552 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.695541 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa0c6c0-3b7b-4748-b5ec-067e5115eb57" containerName="authorino" Jul 09 14:07:56.700898 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.700870 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:07:56.703662 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.703638 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 14:07:56.703807 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.703638 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t2lmj\"" Jul 09 14:07:56.703807 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.703639 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 14:07:56.710394 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.710367 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d4777695c-mmfhr"] Jul 09 14:07:56.798986 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.798921 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5774630-24af-4269-82fe-43ceb92e19c7-webhook-cert\") pod \"maas-controller-7d4777695c-mmfhr\" (UID: \"d5774630-24af-4269-82fe-43ceb92e19c7\") " pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:07:56.798986 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.798970 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmklv\" (UniqueName: \"kubernetes.io/projected/d5774630-24af-4269-82fe-43ceb92e19c7-kube-api-access-hmklv\") pod \"maas-controller-7d4777695c-mmfhr\" (UID: \"d5774630-24af-4269-82fe-43ceb92e19c7\") " pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:07:56.860603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.860566 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6864bb696c-rmv9n"] Jul 09 14:07:56.863197 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.863176 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:07:56.874824 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.874792 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6864bb696c-rmv9n"] Jul 09 14:07:56.899525 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.899482 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5774630-24af-4269-82fe-43ceb92e19c7-webhook-cert\") pod \"maas-controller-7d4777695c-mmfhr\" (UID: \"d5774630-24af-4269-82fe-43ceb92e19c7\") " pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:07:56.899719 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.899534 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmklv\" (UniqueName: \"kubernetes.io/projected/d5774630-24af-4269-82fe-43ceb92e19c7-kube-api-access-hmklv\") pod \"maas-controller-7d4777695c-mmfhr\" (UID: \"d5774630-24af-4269-82fe-43ceb92e19c7\") " pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:07:56.902191 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.902168 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5774630-24af-4269-82fe-43ceb92e19c7-webhook-cert\") pod \"maas-controller-7d4777695c-mmfhr\" (UID: \"d5774630-24af-4269-82fe-43ceb92e19c7\") " pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:07:56.909899 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:56.909867 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmklv\" (UniqueName: \"kubernetes.io/projected/d5774630-24af-4269-82fe-43ceb92e19c7-kube-api-access-hmklv\") pod \"maas-controller-7d4777695c-mmfhr\" (UID: \"d5774630-24af-4269-82fe-43ceb92e19c7\") " pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:07:57.001175 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.001079 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47b2312e-6f3d-4e20-996d-20ee56e8050e-webhook-cert\") pod \"maas-controller-6864bb696c-rmv9n\" (UID: \"47b2312e-6f3d-4e20-996d-20ee56e8050e\") " pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:07:57.001175 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.001154 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnm58\" (UniqueName: \"kubernetes.io/projected/47b2312e-6f3d-4e20-996d-20ee56e8050e-kube-api-access-qnm58\") pod \"maas-controller-6864bb696c-rmv9n\" (UID: \"47b2312e-6f3d-4e20-996d-20ee56e8050e\") " pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:07:57.014062 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.014033 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:07:57.101756 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.101679 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47b2312e-6f3d-4e20-996d-20ee56e8050e-webhook-cert\") pod \"maas-controller-6864bb696c-rmv9n\" (UID: \"47b2312e-6f3d-4e20-996d-20ee56e8050e\") " pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:07:57.101983 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.101771 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnm58\" (UniqueName: \"kubernetes.io/projected/47b2312e-6f3d-4e20-996d-20ee56e8050e-kube-api-access-qnm58\") pod \"maas-controller-6864bb696c-rmv9n\" (UID: \"47b2312e-6f3d-4e20-996d-20ee56e8050e\") " pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:07:57.104454 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.104423 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47b2312e-6f3d-4e20-996d-20ee56e8050e-webhook-cert\") pod \"maas-controller-6864bb696c-rmv9n\" (UID: \"47b2312e-6f3d-4e20-996d-20ee56e8050e\") " pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:07:57.112099 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.112062 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnm58\" (UniqueName: \"kubernetes.io/projected/47b2312e-6f3d-4e20-996d-20ee56e8050e-kube-api-access-qnm58\") pod \"maas-controller-6864bb696c-rmv9n\" (UID: \"47b2312e-6f3d-4e20-996d-20ee56e8050e\") " pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:07:57.174289 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.173598 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:07:57.179604 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.179537 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d4777695c-mmfhr"] Jul 09 14:07:57.180125 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:07:57.180089 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5774630_24af_4269_82fe_43ceb92e19c7.slice/crio-551c38726737e3d3f9e099cd819a782c7e520e0e6e997bd6b86f3c43697cc109 WatchSource:0}: Error finding container 551c38726737e3d3f9e099cd819a782c7e520e0e6e997bd6b86f3c43697cc109: Status 404 returned error can't find the container with id 551c38726737e3d3f9e099cd819a782c7e520e0e6e997bd6b86f3c43697cc109 Jul 09 14:07:57.334738 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.334707 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6864bb696c-rmv9n"] Jul 09 14:07:57.334950 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:07:57.334927 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b2312e_6f3d_4e20_996d_20ee56e8050e.slice/crio-c0866a0c1015c2d48674ad377ae68900fe405ba545e0e6475bbf4f0df0466bae WatchSource:0}: Error finding container c0866a0c1015c2d48674ad377ae68900fe405ba545e0e6475bbf4f0df0466bae: Status 404 returned error can't find the container with id c0866a0c1015c2d48674ad377ae68900fe405ba545e0e6475bbf4f0df0466bae Jul 09 14:07:57.856142 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.856098 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6864bb696c-rmv9n" event={"ID":"47b2312e-6f3d-4e20-996d-20ee56e8050e","Type":"ContainerStarted","Data":"c0866a0c1015c2d48674ad377ae68900fe405ba545e0e6475bbf4f0df0466bae"} Jul 09 14:07:57.857330 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:07:57.857308 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d4777695c-mmfhr" event={"ID":"d5774630-24af-4269-82fe-43ceb92e19c7","Type":"ContainerStarted","Data":"551c38726737e3d3f9e099cd819a782c7e520e0e6e997bd6b86f3c43697cc109"} Jul 09 14:08:01.883976 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:01.883933 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6864bb696c-rmv9n" event={"ID":"47b2312e-6f3d-4e20-996d-20ee56e8050e","Type":"ContainerStarted","Data":"36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568"} Jul 09 14:08:01.884523 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:01.884014 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:08:01.885687 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:01.885655 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d4777695c-mmfhr" event={"ID":"d5774630-24af-4269-82fe-43ceb92e19c7","Type":"ContainerStarted","Data":"f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa"} Jul 09 14:08:01.885801 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:01.885710 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:08:01.904009 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:01.903950 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6864bb696c-rmv9n" podStartSLOduration=2.340982655 podStartE2EDuration="5.903935107s" totalImagesPullingTime="3.562952452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:01.902760365 +0000 UTC m=+745.756711841" watchObservedRunningTime="2026-07-09 14:08:01.903935107 +0000 UTC m=+745.757886594" Jul 09 14:08:01.929765 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:01.929711 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d4777695c-mmfhr" podStartSLOduration=2.223406339 podStartE2EDuration="5.929692562s" totalImagesPullingTime="3.706286223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:01.928150905 +0000 UTC m=+745.782102381" watchObservedRunningTime="2026-07-09 14:08:01.929692562 +0000 UTC m=+745.783644038" Jul 09 14:08:12.896892 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:12.896837 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:08:12.897328 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:12.897030 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:08:12.957369 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:12.957336 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d4777695c-mmfhr"] Jul 09 14:08:12.957600 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:12.957571 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7d4777695c-mmfhr" podUID="d5774630-24af-4269-82fe-43ceb92e19c7" containerName="manager" containerID="cri-o://f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa" gracePeriod=30 Jul 09 14:08:13.211573 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.211544 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:08:13.251347 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.251308 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmklv\" (UniqueName: \"kubernetes.io/projected/d5774630-24af-4269-82fe-43ceb92e19c7-kube-api-access-hmklv\") pod \"d5774630-24af-4269-82fe-43ceb92e19c7\" (UID: \"d5774630-24af-4269-82fe-43ceb92e19c7\") " Jul 09 14:08:13.251513 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.251398 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5774630-24af-4269-82fe-43ceb92e19c7-webhook-cert\") pod \"d5774630-24af-4269-82fe-43ceb92e19c7\" (UID: \"d5774630-24af-4269-82fe-43ceb92e19c7\") " Jul 09 14:08:13.253651 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.253617 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5774630-24af-4269-82fe-43ceb92e19c7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d5774630-24af-4269-82fe-43ceb92e19c7" (UID: "d5774630-24af-4269-82fe-43ceb92e19c7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:08:13.253811 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.253747 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5774630-24af-4269-82fe-43ceb92e19c7-kube-api-access-hmklv" (OuterVolumeSpecName: "kube-api-access-hmklv") pod "d5774630-24af-4269-82fe-43ceb92e19c7" (UID: "d5774630-24af-4269-82fe-43ceb92e19c7"). InnerVolumeSpecName "kube-api-access-hmklv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:08:13.352732 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.352634 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmklv\" (UniqueName: \"kubernetes.io/projected/d5774630-24af-4269-82fe-43ceb92e19c7-kube-api-access-hmklv\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:08:13.352732 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.352728 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5774630-24af-4269-82fe-43ceb92e19c7-webhook-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:08:13.955174 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.955139 2575 generic.go:356] "Generic (PLEG): container finished" podID="d5774630-24af-4269-82fe-43ceb92e19c7" containerID="f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa" exitCode=0 Jul 09 14:08:13.955671 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.955192 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d4777695c-mmfhr" Jul 09 14:08:13.955671 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.955234 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d4777695c-mmfhr" event={"ID":"d5774630-24af-4269-82fe-43ceb92e19c7","Type":"ContainerDied","Data":"f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa"} Jul 09 14:08:13.955671 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.955299 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d4777695c-mmfhr" event={"ID":"d5774630-24af-4269-82fe-43ceb92e19c7","Type":"ContainerDied","Data":"551c38726737e3d3f9e099cd819a782c7e520e0e6e997bd6b86f3c43697cc109"} Jul 09 14:08:13.955671 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.955324 2575 scope.go:117] "RemoveContainer" containerID="f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa" Jul 09 14:08:13.971412 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.971293 2575 scope.go:117] "RemoveContainer" containerID="f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa" Jul 09 14:08:13.971651 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:08:13.971632 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa\": container with ID starting with f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa not found: ID does not exist" containerID="f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa" Jul 09 14:08:13.971711 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.971657 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa"} err="failed to get container status \"f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa\": rpc error: code = NotFound desc = could not find container \"f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa\": container with ID starting with f6e4fd670dfcc7b582a7c303ab47e440b58fadc8420c3fd83cab027d39662aaa not found: ID does not exist" Jul 09 14:08:13.981447 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.981411 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d4777695c-mmfhr"] Jul 09 14:08:13.984778 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:13.984747 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d4777695c-mmfhr"] Jul 09 14:08:14.637783 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:14.637746 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5774630-24af-4269-82fe-43ceb92e19c7" path="/var/lib/kubelet/pods/d5774630-24af-4269-82fe-43ceb92e19c7/volumes" Jul 09 14:08:37.329268 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.329225 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-dp7mm"] Jul 09 14:08:37.330019 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.329995 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5774630-24af-4269-82fe-43ceb92e19c7" containerName="manager" Jul 09 14:08:37.330019 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.330021 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5774630-24af-4269-82fe-43ceb92e19c7" containerName="manager" Jul 09 14:08:37.330211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.330116 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5774630-24af-4269-82fe-43ceb92e19c7" containerName="manager" Jul 09 14:08:37.333437 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.333412 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:37.336170 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.336144 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 14:08:37.336311 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.336285 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2cmp7\"" Jul 09 14:08:37.342972 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.342932 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5lpjj"] Jul 09 14:08:37.346235 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.346204 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-dp7mm"] Jul 09 14:08:37.346365 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.346315 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:37.357458 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.357430 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5lpjj"] Jul 09 14:08:37.483717 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.483679 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z75bn\" (UniqueName: \"kubernetes.io/projected/073f64b8-36fc-4f7e-9999-238a9c5ebb87-kube-api-access-z75bn\") pod \"payload-pre-processing-7964457696-dp7mm\" (UID: \"073f64b8-36fc-4f7e-9999-238a9c5ebb87\") " pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:37.483951 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.483741 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b882159-647a-4daa-a7c7-24af8297d013-plugins-config-volume\") pod \"payload-processing-58f86c7798-5lpjj\" (UID: \"5b882159-647a-4daa-a7c7-24af8297d013\") " pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:37.483951 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.483762 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqkcp\" (UniqueName: \"kubernetes.io/projected/5b882159-647a-4daa-a7c7-24af8297d013-kube-api-access-rqkcp\") pod \"payload-processing-58f86c7798-5lpjj\" (UID: \"5b882159-647a-4daa-a7c7-24af8297d013\") " pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:37.483951 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.483808 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/073f64b8-36fc-4f7e-9999-238a9c5ebb87-plugins-config-volume\") pod \"payload-pre-processing-7964457696-dp7mm\" (UID: \"073f64b8-36fc-4f7e-9999-238a9c5ebb87\") " pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:37.585109 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.585004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b882159-647a-4daa-a7c7-24af8297d013-plugins-config-volume\") pod \"payload-processing-58f86c7798-5lpjj\" (UID: \"5b882159-647a-4daa-a7c7-24af8297d013\") " pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:37.585109 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.585046 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqkcp\" (UniqueName: \"kubernetes.io/projected/5b882159-647a-4daa-a7c7-24af8297d013-kube-api-access-rqkcp\") pod \"payload-processing-58f86c7798-5lpjj\" (UID: \"5b882159-647a-4daa-a7c7-24af8297d013\") " pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:37.585109 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.585078 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/073f64b8-36fc-4f7e-9999-238a9c5ebb87-plugins-config-volume\") pod \"payload-pre-processing-7964457696-dp7mm\" (UID: \"073f64b8-36fc-4f7e-9999-238a9c5ebb87\") " pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:37.585424 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.585153 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z75bn\" (UniqueName: \"kubernetes.io/projected/073f64b8-36fc-4f7e-9999-238a9c5ebb87-kube-api-access-z75bn\") pod \"payload-pre-processing-7964457696-dp7mm\" (UID: \"073f64b8-36fc-4f7e-9999-238a9c5ebb87\") " pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:37.585926 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.585906 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b882159-647a-4daa-a7c7-24af8297d013-plugins-config-volume\") pod \"payload-processing-58f86c7798-5lpjj\" (UID: \"5b882159-647a-4daa-a7c7-24af8297d013\") " pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:37.585973 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.585922 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/073f64b8-36fc-4f7e-9999-238a9c5ebb87-plugins-config-volume\") pod \"payload-pre-processing-7964457696-dp7mm\" (UID: \"073f64b8-36fc-4f7e-9999-238a9c5ebb87\") " pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:37.594226 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.594197 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqkcp\" (UniqueName: \"kubernetes.io/projected/5b882159-647a-4daa-a7c7-24af8297d013-kube-api-access-rqkcp\") pod \"payload-processing-58f86c7798-5lpjj\" (UID: \"5b882159-647a-4daa-a7c7-24af8297d013\") " pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:37.594386 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.594331 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z75bn\" (UniqueName: \"kubernetes.io/projected/073f64b8-36fc-4f7e-9999-238a9c5ebb87-kube-api-access-z75bn\") pod \"payload-pre-processing-7964457696-dp7mm\" (UID: \"073f64b8-36fc-4f7e-9999-238a9c5ebb87\") " pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:37.645264 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.645220 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:37.658683 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.658649 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:37.822756 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.822721 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-dp7mm"] Jul 09 14:08:37.825823 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:08:37.825795 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073f64b8_36fc_4f7e_9999_238a9c5ebb87.slice/crio-1de97c0b88590658351017e15fa2ddd8ae6520426d15854660ce5807709f6110 WatchSource:0}: Error finding container 1de97c0b88590658351017e15fa2ddd8ae6520426d15854660ce5807709f6110: Status 404 returned error can't find the container with id 1de97c0b88590658351017e15fa2ddd8ae6520426d15854660ce5807709f6110 Jul 09 14:08:37.850467 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:37.850437 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5lpjj"] Jul 09 14:08:37.852677 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:08:37.852651 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b882159_647a_4daa_a7c7_24af8297d013.slice/crio-8887a4d1fbfead288d6ffdf8b3b88f2a15866e555606acd9550a86f35a852f30 WatchSource:0}: Error finding container 8887a4d1fbfead288d6ffdf8b3b88f2a15866e555606acd9550a86f35a852f30: Status 404 returned error can't find the container with id 8887a4d1fbfead288d6ffdf8b3b88f2a15866e555606acd9550a86f35a852f30 Jul 09 14:08:38.090391 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:38.090356 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" event={"ID":"5b882159-647a-4daa-a7c7-24af8297d013","Type":"ContainerStarted","Data":"8887a4d1fbfead288d6ffdf8b3b88f2a15866e555606acd9550a86f35a852f30"} Jul 09 14:08:38.091907 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:38.091878 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" event={"ID":"073f64b8-36fc-4f7e-9999-238a9c5ebb87","Type":"ContainerStarted","Data":"1de97c0b88590658351017e15fa2ddd8ae6520426d15854660ce5807709f6110"} Jul 09 14:08:40.114367 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:40.114322 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" event={"ID":"5b882159-647a-4daa-a7c7-24af8297d013","Type":"ContainerStarted","Data":"f3adb8e40479816738a77ebc61e712a0ba20565783d286948f5f9eca8d779d34"} Jul 09 14:08:40.114816 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:40.114398 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:40.116244 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:40.116219 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" event={"ID":"073f64b8-36fc-4f7e-9999-238a9c5ebb87","Type":"ContainerStarted","Data":"74299ec23aba3770ea898eddd9711c854b52e8587356d4358a7ca754906130bf"} Jul 09 14:08:40.116391 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:40.116273 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:40.132475 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:40.132413 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" podStartSLOduration=1.654948555 podStartE2EDuration="3.132395709s" totalImagesPullingTime="1.477447154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:40.131645098 +0000 UTC m=+783.985596587" watchObservedRunningTime="2026-07-09 14:08:40.132395709 +0000 UTC m=+783.986347185" Jul 09 14:08:40.149546 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:40.149487 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" podStartSLOduration=1.6425459999999998 podStartE2EDuration="3.149467528s" totalImagesPullingTime="1.506921528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:08:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:40.147960983 +0000 UTC m=+784.001912460" watchObservedRunningTime="2026-07-09 14:08:40.149467528 +0000 UTC m=+784.003419004" Jul 09 14:08:46.150409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:46.150372 2575 generic.go:356] "Generic (PLEG): container finished" podID="08e20f32-3d40-470c-b35f-7b288fb5bb66" containerID="da9b3c8973bfbc57a4e689ea353d9384f802059adf21908722e75bb3a9bc0ba3" exitCode=0 Jul 09 14:08:46.150988 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:46.150449 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" event={"ID":"08e20f32-3d40-470c-b35f-7b288fb5bb66","Type":"ContainerDied","Data":"da9b3c8973bfbc57a4e689ea353d9384f802059adf21908722e75bb3a9bc0ba3"} Jul 09 14:08:46.151079 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:46.151010 2575 scope.go:117] "RemoveContainer" containerID="da9b3c8973bfbc57a4e689ea353d9384f802059adf21908722e75bb3a9bc0ba3" Jul 09 14:08:46.152186 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:46.152101 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:08:46.152262 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:46.152230 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:08:46.152303 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:46.152266 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:08:47.158915 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:47.158880 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" event={"ID":"08e20f32-3d40-470c-b35f-7b288fb5bb66","Type":"ContainerStarted","Data":"2ac6921178412ccecc5e3c57a35f6735de5767634ef67ee2dcb68d14e8fc85e1"} Jul 09 14:08:47.617999 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:47.617952 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:08:47.822304 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:47.822268 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:08:47.823720 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:47.823691 2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2njfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:08:47.823874 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:47.823751 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" podUID="08e20f32-3d40-470c-b35f-7b288fb5bb66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 09 14:08:48.822520 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:48.822485 2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2njfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:08:48.822957 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:48.822543 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" podUID="08e20f32-3d40-470c-b35f-7b288fb5bb66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 09 14:08:49.822507 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:49.822470 2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2njfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:08:49.822700 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:49.822535 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" podUID="08e20f32-3d40-470c-b35f-7b288fb5bb66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 09 14:08:50.822945 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:50.822904 2575 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-2njfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 09 14:08:50.823379 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:50.822962 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" podUID="08e20f32-3d40-470c-b35f-7b288fb5bb66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 09 14:08:51.124101 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:51.124002 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-5lpjj" Jul 09 14:08:51.124101 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:51.124093 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-dp7mm" Jul 09 14:08:51.826493 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:51.826462 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:08:51.827507 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:08:51.827489 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-2njfs" Jul 09 14:09:24.638131 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.638103 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7"] Jul 09 14:09:24.642895 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.642876 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.645799 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.645766 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 14:09:24.645970 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.645905 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zc5jh\"" Jul 09 14:09:24.645970 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.645910 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 14:09:24.646102 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.646048 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 14:09:24.648046 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.648021 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7"] Jul 09 14:09:24.747118 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.747077 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.747293 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.747143 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btbtb\" (UniqueName: \"kubernetes.io/projected/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-kube-api-access-btbtb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.747293 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.747215 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.747293 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.747263 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.747293 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.747277 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.747434 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.747299 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.848809 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.848768 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.849045 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.848837 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.849045 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.848884 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.849045 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.848913 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.849045 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.848971 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.849045 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.849011 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btbtb\" (UniqueName: \"kubernetes.io/projected/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-kube-api-access-btbtb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.849333 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.849306 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.849454 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.849337 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.849615 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.849534 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.851428 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.851406 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.851516 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.851468 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.857346 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.857324 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btbtb\" (UniqueName: \"kubernetes.io/projected/a37f0ded-f0b7-4851-acfb-cec15d4f1a99-kube-api-access-btbtb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-t26l7\" (UID: \"a37f0ded-f0b7-4851-acfb-cec15d4f1a99\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:24.955743 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:24.955649 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:25.105316 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:25.102623 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7"] Jul 09 14:09:25.397584 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:25.397550 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" event={"ID":"a37f0ded-f0b7-4851-acfb-cec15d4f1a99","Type":"ContainerStarted","Data":"a848d312e906f0c5cff5bf34cd32b6c384529009029533406fc7a11d3a8dcea4"} Jul 09 14:09:26.404948 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:26.404916 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" event={"ID":"a37f0ded-f0b7-4851-acfb-cec15d4f1a99","Type":"ContainerStarted","Data":"50f48529f5f6c53710b031817739441a538805528b892404b989d3516d18c33d"} Jul 09 14:09:26.423787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:26.423720 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" podStartSLOduration=1.230245064 podStartE2EDuration="2.423699785s" totalImagesPullingTime="1.193454721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:26.421465012 +0000 UTC m=+830.275416486" watchObservedRunningTime="2026-07-09 14:09:26.423699785 +0000 UTC m=+830.277651261" Jul 09 14:09:34.955800 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:34.955750 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:34.956253 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:34.955822 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:34.968734 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:34.968703 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:35.467061 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:35.467026 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-t26l7" Jul 09 14:09:44.651406 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.651370 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59"] Jul 09 14:09:44.655318 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.655297 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.658076 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.658052 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 09 14:09:44.669129 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.669102 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59"] Jul 09 14:09:44.727992 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.727954 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4778066a-e22b-4ef8-b641-d965eb648d8c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.728188 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.728012 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.728188 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.728089 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.728188 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.728104 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.728188 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.728153 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.728350 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.728194 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgbs7\" (UniqueName: \"kubernetes.io/projected/4778066a-e22b-4ef8-b641-d965eb648d8c-kube-api-access-mgbs7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.829652 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.829610 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4778066a-e22b-4ef8-b641-d965eb648d8c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.829840 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.829659 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.829840 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.829702 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.829840 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.829719 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.829840 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.829743 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.829840 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.829784 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgbs7\" (UniqueName: \"kubernetes.io/projected/4778066a-e22b-4ef8-b641-d965eb648d8c-kube-api-access-mgbs7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.830276 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.830251 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.830276 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.830275 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.830643 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.830326 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.832167 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.832144 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4778066a-e22b-4ef8-b641-d965eb648d8c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.832409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.832391 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4778066a-e22b-4ef8-b641-d965eb648d8c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.837920 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.837898 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgbs7\" (UniqueName: \"kubernetes.io/projected/4778066a-e22b-4ef8-b641-d965eb648d8c-kube-api-access-mgbs7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59\" (UID: \"4778066a-e22b-4ef8-b641-d965eb648d8c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:44.966177 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:44.966086 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:45.107831 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:45.107807 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59"] Jul 09 14:09:45.109336 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:09:45.109311 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4778066a_e22b_4ef8_b641_d965eb648d8c.slice/crio-da314648b349b7b210abf8a7861fffa5c8d2edebca1237639fd9ff146f21b1b5 WatchSource:0}: Error finding container da314648b349b7b210abf8a7861fffa5c8d2edebca1237639fd9ff146f21b1b5: Status 404 returned error can't find the container with id da314648b349b7b210abf8a7861fffa5c8d2edebca1237639fd9ff146f21b1b5 Jul 09 14:09:45.514297 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:45.514262 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" event={"ID":"4778066a-e22b-4ef8-b641-d965eb648d8c","Type":"ContainerStarted","Data":"2bfdc2e2116e1e33f70c64f179b4ad07cff762691013e50405a13b5c5944531f"} Jul 09 14:09:45.514478 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:45.514320 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" event={"ID":"4778066a-e22b-4ef8-b641-d965eb648d8c","Type":"ContainerStarted","Data":"da314648b349b7b210abf8a7861fffa5c8d2edebca1237639fd9ff146f21b1b5"} Jul 09 14:09:45.538657 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:45.538604 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" podStartSLOduration=1.299710799 podStartE2EDuration="1.538586118s" totalImagesPullingTime="238.875319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:09:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:09:45.536736505 +0000 UTC m=+849.390687982" watchObservedRunningTime="2026-07-09 14:09:45.538586118 +0000 UTC m=+849.392537628" Jul 09 14:09:54.967678 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:54.967642 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:54.968208 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:54.967716 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:54.980557 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:54.980529 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:09:55.588024 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:09:55.587984 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59" Jul 09 14:11:03.850562 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.850530 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-786779555b-cwfxk"] Jul 09 14:11:03.854082 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.854065 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-786779555b-cwfxk" Jul 09 14:11:03.856955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.856932 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-85j9k\"" Jul 09 14:11:03.858309 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.858287 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 14:11:03.863971 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.863946 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-786779555b-cwfxk"] Jul 09 14:11:03.886384 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.886337 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xzkg\" (UniqueName: \"kubernetes.io/projected/3118e8fc-e111-4e21-b3c1-912e37a282d3-kube-api-access-2xzkg\") pod \"authorino-786779555b-cwfxk\" (UID: \"3118e8fc-e111-4e21-b3c1-912e37a282d3\") " pod="kuadrant-system/authorino-786779555b-cwfxk" Jul 09 14:11:03.886567 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.886402 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3118e8fc-e111-4e21-b3c1-912e37a282d3-tls-cert\") pod \"authorino-786779555b-cwfxk\" (UID: \"3118e8fc-e111-4e21-b3c1-912e37a282d3\") " pod="kuadrant-system/authorino-786779555b-cwfxk" Jul 09 14:11:03.987238 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.987205 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xzkg\" (UniqueName: \"kubernetes.io/projected/3118e8fc-e111-4e21-b3c1-912e37a282d3-kube-api-access-2xzkg\") pod \"authorino-786779555b-cwfxk\" (UID: \"3118e8fc-e111-4e21-b3c1-912e37a282d3\") " pod="kuadrant-system/authorino-786779555b-cwfxk" Jul 09 14:11:03.987434 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.987247 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3118e8fc-e111-4e21-b3c1-912e37a282d3-tls-cert\") pod \"authorino-786779555b-cwfxk\" (UID: \"3118e8fc-e111-4e21-b3c1-912e37a282d3\") " pod="kuadrant-system/authorino-786779555b-cwfxk" Jul 09 14:11:03.989915 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.989896 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3118e8fc-e111-4e21-b3c1-912e37a282d3-tls-cert\") pod \"authorino-786779555b-cwfxk\" (UID: \"3118e8fc-e111-4e21-b3c1-912e37a282d3\") " pod="kuadrant-system/authorino-786779555b-cwfxk" Jul 09 14:11:03.995157 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:03.995133 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xzkg\" (UniqueName: \"kubernetes.io/projected/3118e8fc-e111-4e21-b3c1-912e37a282d3-kube-api-access-2xzkg\") pod \"authorino-786779555b-cwfxk\" (UID: \"3118e8fc-e111-4e21-b3c1-912e37a282d3\") " pod="kuadrant-system/authorino-786779555b-cwfxk" Jul 09 14:11:04.164135 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:04.164051 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-786779555b-cwfxk" Jul 09 14:11:04.311304 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:04.311269 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-786779555b-cwfxk"] Jul 09 14:11:04.312931 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:11:04.312902 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3118e8fc_e111_4e21_b3c1_912e37a282d3.slice/crio-c5c0e675ca444281707e2b53325654b5b2419d1ccb97c824c452e73d5909029f WatchSource:0}: Error finding container c5c0e675ca444281707e2b53325654b5b2419d1ccb97c824c452e73d5909029f: Status 404 returned error can't find the container with id c5c0e675ca444281707e2b53325654b5b2419d1ccb97c824c452e73d5909029f Jul 09 14:11:04.314227 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:04.314208 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 14:11:04.976827 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:04.976782 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-786779555b-cwfxk" event={"ID":"3118e8fc-e111-4e21-b3c1-912e37a282d3","Type":"ContainerStarted","Data":"c5c0e675ca444281707e2b53325654b5b2419d1ccb97c824c452e73d5909029f"} Jul 09 14:11:05.985559 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:05.985526 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-786779555b-cwfxk" event={"ID":"3118e8fc-e111-4e21-b3c1-912e37a282d3","Type":"ContainerStarted","Data":"f824b3b4bf55b8c50dfab742f251c1c0bb1e694fd0a5cefcf16110cb77f8f00b"} Jul 09 14:11:06.004623 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:11:06.004564 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-786779555b-cwfxk" podStartSLOduration=2.01276755 podStartE2EDuration="3.004548439s" totalImagesPullingTime="991.780889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:11:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:11:06.001965688 +0000 UTC m=+929.855917165" watchObservedRunningTime="2026-07-09 14:11:06.004548439 +0000 UTC m=+929.858499915" Jul 09 14:12:18.146238 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.146193 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6864bb696c-rmv9n"] Jul 09 14:12:18.146769 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.146427 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6864bb696c-rmv9n" podUID="47b2312e-6f3d-4e20-996d-20ee56e8050e" containerName="manager" containerID="cri-o://36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568" gracePeriod=30 Jul 09 14:12:18.398884 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.398809 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:12:18.405938 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.405902 2575 generic.go:356] "Generic (PLEG): container finished" podID="47b2312e-6f3d-4e20-996d-20ee56e8050e" containerID="36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568" exitCode=0 Jul 09 14:12:18.405938 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.405939 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6864bb696c-rmv9n" Jul 09 14:12:18.406136 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.405989 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6864bb696c-rmv9n" event={"ID":"47b2312e-6f3d-4e20-996d-20ee56e8050e","Type":"ContainerDied","Data":"36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568"} Jul 09 14:12:18.406136 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.406080 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6864bb696c-rmv9n" event={"ID":"47b2312e-6f3d-4e20-996d-20ee56e8050e","Type":"ContainerDied","Data":"c0866a0c1015c2d48674ad377ae68900fe405ba545e0e6475bbf4f0df0466bae"} Jul 09 14:12:18.406136 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.406112 2575 scope.go:117] "RemoveContainer" containerID="36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568" Jul 09 14:12:18.415320 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.415300 2575 scope.go:117] "RemoveContainer" containerID="36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568" Jul 09 14:12:18.415613 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:12:18.415594 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568\": container with ID starting with 36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568 not found: ID does not exist" containerID="36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568" Jul 09 14:12:18.415663 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.415625 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568"} err="failed to get container status \"36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568\": rpc error: code = NotFound desc = could not find container \"36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568\": container with ID starting with 36337b4643e3649ce5006ca94419f2339eb985e6f93f1fff2faece714cb97568 not found: ID does not exist" Jul 09 14:12:18.502831 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.502788 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnm58\" (UniqueName: \"kubernetes.io/projected/47b2312e-6f3d-4e20-996d-20ee56e8050e-kube-api-access-qnm58\") pod \"47b2312e-6f3d-4e20-996d-20ee56e8050e\" (UID: \"47b2312e-6f3d-4e20-996d-20ee56e8050e\") " Jul 09 14:12:18.502831 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.502836 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47b2312e-6f3d-4e20-996d-20ee56e8050e-webhook-cert\") pod \"47b2312e-6f3d-4e20-996d-20ee56e8050e\" (UID: \"47b2312e-6f3d-4e20-996d-20ee56e8050e\") " Jul 09 14:12:18.505236 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.505200 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47b2312e-6f3d-4e20-996d-20ee56e8050e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "47b2312e-6f3d-4e20-996d-20ee56e8050e" (UID: "47b2312e-6f3d-4e20-996d-20ee56e8050e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 14:12:18.505236 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.505214 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b2312e-6f3d-4e20-996d-20ee56e8050e-kube-api-access-qnm58" (OuterVolumeSpecName: "kube-api-access-qnm58") pod "47b2312e-6f3d-4e20-996d-20ee56e8050e" (UID: "47b2312e-6f3d-4e20-996d-20ee56e8050e"). InnerVolumeSpecName "kube-api-access-qnm58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:12:18.603919 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.603873 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnm58\" (UniqueName: \"kubernetes.io/projected/47b2312e-6f3d-4e20-996d-20ee56e8050e-kube-api-access-qnm58\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:12:18.603919 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.603908 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/47b2312e-6f3d-4e20-996d-20ee56e8050e-webhook-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:12:18.723764 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.723674 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6864bb696c-rmv9n"] Jul 09 14:12:18.728075 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:18.728044 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6864bb696c-rmv9n"] Jul 09 14:12:19.499438 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.499400 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6864bb696c-h54jr"] Jul 09 14:12:19.499837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.499830 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b2312e-6f3d-4e20-996d-20ee56e8050e" containerName="manager" Jul 09 14:12:19.499905 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.499841 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b2312e-6f3d-4e20-996d-20ee56e8050e" containerName="manager" Jul 09 14:12:19.499967 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.499955 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="47b2312e-6f3d-4e20-996d-20ee56e8050e" containerName="manager" Jul 09 14:12:19.504322 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.504305 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:12:19.508424 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.508360 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 14:12:19.508424 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.508360 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 14:12:19.508424 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.508361 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t2lmj\"" Jul 09 14:12:19.512422 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.512009 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6864bb696c-h54jr"] Jul 09 14:12:19.611974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.611935 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm46j\" (UniqueName: \"kubernetes.io/projected/ce2dd036-027e-4656-94a6-439c4740cf8d-kube-api-access-pm46j\") pod \"maas-controller-6864bb696c-h54jr\" (UID: \"ce2dd036-027e-4656-94a6-439c4740cf8d\") " pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:12:19.612175 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.611993 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce2dd036-027e-4656-94a6-439c4740cf8d-webhook-cert\") pod \"maas-controller-6864bb696c-h54jr\" (UID: \"ce2dd036-027e-4656-94a6-439c4740cf8d\") " pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:12:19.712496 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.712444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm46j\" (UniqueName: \"kubernetes.io/projected/ce2dd036-027e-4656-94a6-439c4740cf8d-kube-api-access-pm46j\") pod \"maas-controller-6864bb696c-h54jr\" (UID: \"ce2dd036-027e-4656-94a6-439c4740cf8d\") " pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:12:19.712687 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.712516 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce2dd036-027e-4656-94a6-439c4740cf8d-webhook-cert\") pod \"maas-controller-6864bb696c-h54jr\" (UID: \"ce2dd036-027e-4656-94a6-439c4740cf8d\") " pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:12:19.715170 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.715148 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce2dd036-027e-4656-94a6-439c4740cf8d-webhook-cert\") pod \"maas-controller-6864bb696c-h54jr\" (UID: \"ce2dd036-027e-4656-94a6-439c4740cf8d\") " pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:12:19.721895 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.721834 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm46j\" (UniqueName: \"kubernetes.io/projected/ce2dd036-027e-4656-94a6-439c4740cf8d-kube-api-access-pm46j\") pod \"maas-controller-6864bb696c-h54jr\" (UID: \"ce2dd036-027e-4656-94a6-439c4740cf8d\") " pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:12:19.816184 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.816148 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:12:19.983875 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:19.983827 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6864bb696c-h54jr"] Jul 09 14:12:19.985804 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:12:19.985772 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2dd036_027e_4656_94a6_439c4740cf8d.slice/crio-cb18ff9b9a37a4024b61b0e50c493ab3b464fbb642fbfbefa53108da18839704 WatchSource:0}: Error finding container cb18ff9b9a37a4024b61b0e50c493ab3b464fbb642fbfbefa53108da18839704: Status 404 returned error can't find the container with id cb18ff9b9a37a4024b61b0e50c493ab3b464fbb642fbfbefa53108da18839704 Jul 09 14:12:20.420649 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:20.420616 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6864bb696c-h54jr" event={"ID":"ce2dd036-027e-4656-94a6-439c4740cf8d","Type":"ContainerStarted","Data":"cd4f098c254673e92160b3b20af0417b259cef0ed46897e20fdf73a59167b158"} Jul 09 14:12:20.420812 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:20.420672 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6864bb696c-h54jr" event={"ID":"ce2dd036-027e-4656-94a6-439c4740cf8d","Type":"ContainerStarted","Data":"cb18ff9b9a37a4024b61b0e50c493ab3b464fbb642fbfbefa53108da18839704"} Jul 09 14:12:20.420812 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:20.420702 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:12:20.438680 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:20.438617 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6864bb696c-h54jr" podStartSLOduration=1.075297607 podStartE2EDuration="1.438599292s" totalImagesPullingTime="363.301685ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:12:20.436543001 +0000 UTC m=+1004.290494477" watchObservedRunningTime="2026-07-09 14:12:20.438599292 +0000 UTC m=+1004.292550789" Jul 09 14:12:20.638914 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:20.638806 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47b2312e-6f3d-4e20-996d-20ee56e8050e" path="/var/lib/kubelet/pods/47b2312e-6f3d-4e20-996d-20ee56e8050e/volumes" Jul 09 14:12:31.440238 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:12:31.440201 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6864bb696c-h54jr" Jul 09 14:24:11.511019 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.510980 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw"] Jul 09 14:24:11.514802 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.514779 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.517842 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.517810 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b1ce403b-openshift-default-dockercfg-9xgnc\"" Jul 09 14:24:11.550811 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.550769 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw"] Jul 09 14:24:11.612946 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.612907 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27151be8-a600-4954-ada8-b404b4de01fc-istio-podinfo\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.613142 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.612956 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27151be8-a600-4954-ada8-b404b4de01fc-istiod-ca-cert\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.613142 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.613023 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-credential-socket\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.613142 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.613070 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-istio-token\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.613142 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.613092 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-data\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.613290 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.613203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vn8h\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-kube-api-access-4vn8h\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.613290 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.613261 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-envoy\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.613290 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.613285 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-socket\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.613385 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.613317 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-certs\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.714762 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.714714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27151be8-a600-4954-ada8-b404b4de01fc-istiod-ca-cert\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.714999 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.714809 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-credential-socket\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.714999 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.714893 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-istio-token\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.714999 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.714921 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-data\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715174 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715007 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vn8h\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-kube-api-access-4vn8h\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715174 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715100 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-envoy\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715174 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715149 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-socket\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715328 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715261 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-certs\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715328 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715305 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-credential-socket\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715328 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715312 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27151be8-a600-4954-ada8-b404b4de01fc-istio-podinfo\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715587 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715563 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-socket\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715658 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715602 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27151be8-a600-4954-ada8-b404b4de01fc-istiod-ca-cert\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715725 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715698 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-certs\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.715946 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.715926 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-data\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.717698 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.717675 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-envoy\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.718214 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.718197 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27151be8-a600-4954-ada8-b404b4de01fc-istio-podinfo\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.731345 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.731311 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-istio-token\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.732464 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.732445 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vn8h\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-kube-api-access-4vn8h\") pod \"e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.825640 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.825593 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:11.978003 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.977978 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw"] Jul 09 14:24:11.979897 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:24:11.979837 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27151be8_a600_4954_ada8_b404b4de01fc.slice/crio-574bcacf18117a6af8aa4db94ddb858c5d28f4fe8d0ddf972dde5abfbafeedc4 WatchSource:0}: Error finding container 574bcacf18117a6af8aa4db94ddb858c5d28f4fe8d0ddf972dde5abfbafeedc4: Status 404 returned error can't find the container with id 574bcacf18117a6af8aa4db94ddb858c5d28f4fe8d0ddf972dde5abfbafeedc4 Jul 09 14:24:11.982165 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.982123 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:24:11.982283 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.982216 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:24:11.982283 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:11.982248 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:24:12.525797 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:12.525752 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" event={"ID":"27151be8-a600-4954-ada8-b404b4de01fc","Type":"ContainerStarted","Data":"eee567fd7c3bf68d1c16bebb9c598ca55a9e4a71e3ae05672f96ad8d1b083a78"} Jul 09 14:24:12.526331 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:12.525818 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" event={"ID":"27151be8-a600-4954-ada8-b404b4de01fc","Type":"ContainerStarted","Data":"574bcacf18117a6af8aa4db94ddb858c5d28f4fe8d0ddf972dde5abfbafeedc4"} Jul 09 14:24:12.559921 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:12.559864 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" podStartSLOduration=1.559833325 podStartE2EDuration="1.559833325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:24:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:24:12.558693326 +0000 UTC m=+1716.412644802" watchObservedRunningTime="2026-07-09 14:24:12.559833325 +0000 UTC m=+1716.413784800" Jul 09 14:24:12.826380 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:12.826297 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:13.830903 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:13.830839 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:14.542752 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:14.542716 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:14.543672 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:14.543648 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:18.013679 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:18.013639 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw"] Jul 09 14:24:18.014218 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:18.013814 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" podUID="27151be8-a600-4954-ada8-b404b4de01fc" containerName="istio-proxy" containerID="cri-o://eee567fd7c3bf68d1c16bebb9c598ca55a9e4a71e3ae05672f96ad8d1b083a78" gracePeriod=30 Jul 09 14:24:23.599696 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.599599 2575 generic.go:356] "Generic (PLEG): container finished" podID="27151be8-a600-4954-ada8-b404b4de01fc" containerID="eee567fd7c3bf68d1c16bebb9c598ca55a9e4a71e3ae05672f96ad8d1b083a78" exitCode=0 Jul 09 14:24:23.599696 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.599673 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" event={"ID":"27151be8-a600-4954-ada8-b404b4de01fc","Type":"ContainerDied","Data":"eee567fd7c3bf68d1c16bebb9c598ca55a9e4a71e3ae05672f96ad8d1b083a78"} Jul 09 14:24:23.676581 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.676553 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:23.751122 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.751033 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27151be8-a600-4954-ada8-b404b4de01fc-istio-podinfo\") pod \"27151be8-a600-4954-ada8-b404b4de01fc\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " Jul 09 14:24:23.751122 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.751070 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-data\") pod \"27151be8-a600-4954-ada8-b404b4de01fc\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " Jul 09 14:24:23.751122 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.751093 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-credential-socket\") pod \"27151be8-a600-4954-ada8-b404b4de01fc\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " Jul 09 14:24:23.751122 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.751111 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27151be8-a600-4954-ada8-b404b4de01fc-istiod-ca-cert\") pod \"27151be8-a600-4954-ada8-b404b4de01fc\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " Jul 09 14:24:23.751409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.751148 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-certs\") pod \"27151be8-a600-4954-ada8-b404b4de01fc\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " Jul 09 14:24:23.751409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.751170 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vn8h\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-kube-api-access-4vn8h\") pod \"27151be8-a600-4954-ada8-b404b4de01fc\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " Jul 09 14:24:23.751409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.751217 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-socket\") pod \"27151be8-a600-4954-ada8-b404b4de01fc\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " Jul 09 14:24:23.751409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.751255 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-istio-token\") pod \"27151be8-a600-4954-ada8-b404b4de01fc\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " Jul 09 14:24:23.752755 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.752736 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-envoy\") pod \"27151be8-a600-4954-ada8-b404b4de01fc\" (UID: \"27151be8-a600-4954-ada8-b404b4de01fc\") " Jul 09 14:24:23.753157 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.753113 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "27151be8-a600-4954-ada8-b404b4de01fc" (UID: "27151be8-a600-4954-ada8-b404b4de01fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:23.753342 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.753314 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "27151be8-a600-4954-ada8-b404b4de01fc" (UID: "27151be8-a600-4954-ada8-b404b4de01fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:23.753454 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.753416 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "27151be8-a600-4954-ada8-b404b4de01fc" (UID: "27151be8-a600-4954-ada8-b404b4de01fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:23.753565 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.753518 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27151be8-a600-4954-ada8-b404b4de01fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "27151be8-a600-4954-ada8-b404b4de01fc" (UID: "27151be8-a600-4954-ada8-b404b4de01fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:24:23.753731 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.753663 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "27151be8-a600-4954-ada8-b404b4de01fc" (UID: "27151be8-a600-4954-ada8-b404b4de01fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:23.755383 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.755359 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27151be8-a600-4954-ada8-b404b4de01fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "27151be8-a600-4954-ada8-b404b4de01fc" (UID: "27151be8-a600-4954-ada8-b404b4de01fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:24:23.755470 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.755445 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-kube-api-access-4vn8h" (OuterVolumeSpecName: "kube-api-access-4vn8h") pod "27151be8-a600-4954-ada8-b404b4de01fc" (UID: "27151be8-a600-4954-ada8-b404b4de01fc"). InnerVolumeSpecName "kube-api-access-4vn8h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:24:23.755679 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.755659 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "27151be8-a600-4954-ada8-b404b4de01fc" (UID: "27151be8-a600-4954-ada8-b404b4de01fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:24:23.755679 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.755669 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "27151be8-a600-4954-ada8-b404b4de01fc" (UID: "27151be8-a600-4954-ada8-b404b4de01fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:23.854151 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.854111 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:23.854151 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.854143 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vn8h\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-kube-api-access-4vn8h\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:23.854151 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.854158 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:23.854420 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.854171 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27151be8-a600-4954-ada8-b404b4de01fc-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:23.854420 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.854184 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:23.854420 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.854194 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27151be8-a600-4954-ada8-b404b4de01fc-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:23.854420 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.854204 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:23.854420 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.854214 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27151be8-a600-4954-ada8-b404b4de01fc-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:23.854420 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:23.854225 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27151be8-a600-4954-ada8-b404b4de01fc-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:24.350374 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.350340 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z"] Jul 09 14:24:24.350864 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.350828 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27151be8-a600-4954-ada8-b404b4de01fc" containerName="istio-proxy" Jul 09 14:24:24.350966 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.350868 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="27151be8-a600-4954-ada8-b404b4de01fc" containerName="istio-proxy" Jul 09 14:24:24.351018 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.350986 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="27151be8-a600-4954-ada8-b404b4de01fc" containerName="istio-proxy" Jul 09 14:24:24.354382 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.354355 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.357246 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.357221 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f4ec3dfc-openshift-default-dockercfg-h446d\"" Jul 09 14:24:24.371214 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.369221 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z"] Jul 09 14:24:24.459377 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.459342 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-token\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.459377 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.459384 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fwrn\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-kube-api-access-5fwrn\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.459633 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.459405 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-podinfo\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.459633 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.459515 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istiod-ca-cert\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.459633 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.459573 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-data\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.459785 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.459642 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-credential-socket\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.459785 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.459677 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-envoy\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.459785 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.459711 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-socket\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.459785 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.459736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-certs\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.560959 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.560920 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fwrn\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-kube-api-access-5fwrn\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.560959 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.560963 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-podinfo\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istiod-ca-cert\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561058 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-data\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561108 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-credential-socket\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561131 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-envoy\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561158 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-socket\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561178 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-certs\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561531 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561214 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-token\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561634 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561599 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-data\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561772 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561636 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-socket\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561782 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-certs\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561947 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561838 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-credential-socket\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.561947 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.561840 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istiod-ca-cert\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.563715 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.563694 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-envoy\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.563801 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.563778 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-podinfo\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.571794 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.571771 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-token\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.572030 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.572008 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fwrn\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-kube-api-access-5fwrn\") pod \"e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.607718 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.607625 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" event={"ID":"27151be8-a600-4954-ada8-b404b4de01fc","Type":"ContainerDied","Data":"574bcacf18117a6af8aa4db94ddb858c5d28f4fe8d0ddf972dde5abfbafeedc4"} Jul 09 14:24:24.607718 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.607655 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw" Jul 09 14:24:24.607718 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.607701 2575 scope.go:117] "RemoveContainer" containerID="eee567fd7c3bf68d1c16bebb9c598ca55a9e4a71e3ae05672f96ad8d1b083a78" Jul 09 14:24:24.646217 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.646181 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw"] Jul 09 14:24:24.660447 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.660418 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b1ce403b-openshift-default-68fbdbd567-4b4gw"] Jul 09 14:24:24.668171 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.668147 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:24.829095 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.829066 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z"] Jul 09 14:24:24.830695 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:24:24.830668 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8e61afa_600c_4a01_a3ab_77b4feb567dd.slice/crio-2a33062b61f74f29072fcc00db3a732ddcecf029251d06523b20c00b583a148e WatchSource:0}: Error finding container 2a33062b61f74f29072fcc00db3a732ddcecf029251d06523b20c00b583a148e: Status 404 returned error can't find the container with id 2a33062b61f74f29072fcc00db3a732ddcecf029251d06523b20c00b583a148e Jul 09 14:24:24.832994 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.832958 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:24:24.833086 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.833034 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:24:24.833086 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:24.833065 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:24:25.616154 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:25.616115 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" event={"ID":"b8e61afa-600c-4a01-a3ab-77b4feb567dd","Type":"ContainerStarted","Data":"397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc"} Jul 09 14:24:25.616667 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:25.616179 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" event={"ID":"b8e61afa-600c-4a01-a3ab-77b4feb567dd","Type":"ContainerStarted","Data":"2a33062b61f74f29072fcc00db3a732ddcecf029251d06523b20c00b583a148e"} Jul 09 14:24:25.652148 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:25.652096 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" podStartSLOduration=1.652077088 podStartE2EDuration="1.652077088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:24:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:24:25.65084105 +0000 UTC m=+1729.504792526" watchObservedRunningTime="2026-07-09 14:24:25.652077088 +0000 UTC m=+1729.506028565" Jul 09 14:24:25.669152 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:25.669115 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:25.675109 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:25.675073 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:26.622716 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:26.622655 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:26.623642 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:26.623622 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:26.638748 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:26.638712 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27151be8-a600-4954-ada8-b404b4de01fc" path="/var/lib/kubelet/pods/27151be8-a600-4954-ada8-b404b4de01fc/volumes" Jul 09 14:24:36.755806 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:36.755757 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z"] Jul 09 14:24:36.756303 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:36.756028 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" podUID="b8e61afa-600c-4a01-a3ab-77b4feb567dd" containerName="istio-proxy" containerID="cri-o://397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc" gracePeriod=30 Jul 09 14:24:38.326601 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.326576 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:38.403488 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403401 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istiod-ca-cert\") pod \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " Jul 09 14:24:38.403488 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403483 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-certs\") pod \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " Jul 09 14:24:38.403722 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403514 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-podinfo\") pod \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " Jul 09 14:24:38.403722 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403548 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-socket\") pod \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " Jul 09 14:24:38.403722 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403593 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-data\") pod \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " Jul 09 14:24:38.403722 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403638 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-credential-socket\") pod \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " Jul 09 14:24:38.403955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403805 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b8e61afa-600c-4a01-a3ab-77b4feb567dd" (UID: "b8e61afa-600c-4a01-a3ab-77b4feb567dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:38.403955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403833 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b8e61afa-600c-4a01-a3ab-77b4feb567dd" (UID: "b8e61afa-600c-4a01-a3ab-77b4feb567dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:24:38.403955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403900 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fwrn\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-kube-api-access-5fwrn\") pod \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " Jul 09 14:24:38.403955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403944 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-token\") pod \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " Jul 09 14:24:38.404166 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.403988 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-envoy\") pod \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\" (UID: \"b8e61afa-600c-4a01-a3ab-77b4feb567dd\") " Jul 09 14:24:38.404463 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.404306 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b8e61afa-600c-4a01-a3ab-77b4feb567dd" (UID: "b8e61afa-600c-4a01-a3ab-77b4feb567dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:38.404463 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.404374 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b8e61afa-600c-4a01-a3ab-77b4feb567dd" (UID: "b8e61afa-600c-4a01-a3ab-77b4feb567dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:38.404662 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.404634 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:38.404662 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.404656 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:38.404781 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.404672 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:38.404781 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.404685 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:38.405089 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.405047 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "b8e61afa-600c-4a01-a3ab-77b4feb567dd" (UID: "b8e61afa-600c-4a01-a3ab-77b4feb567dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:38.406508 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.406484 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b8e61afa-600c-4a01-a3ab-77b4feb567dd" (UID: "b8e61afa-600c-4a01-a3ab-77b4feb567dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:38.406667 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.406634 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b8e61afa-600c-4a01-a3ab-77b4feb567dd" (UID: "b8e61afa-600c-4a01-a3ab-77b4feb567dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:24:38.407161 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.407137 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "b8e61afa-600c-4a01-a3ab-77b4feb567dd" (UID: "b8e61afa-600c-4a01-a3ab-77b4feb567dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:24:38.407236 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.407190 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-kube-api-access-5fwrn" (OuterVolumeSpecName: "kube-api-access-5fwrn") pod "b8e61afa-600c-4a01-a3ab-77b4feb567dd" (UID: "b8e61afa-600c-4a01-a3ab-77b4feb567dd"). InnerVolumeSpecName "kube-api-access-5fwrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:24:38.506238 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.506196 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:38.506238 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.506229 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:38.506238 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.506239 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:38.506238 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.506247 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e61afa-600c-4a01-a3ab-77b4feb567dd-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:38.506518 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.506256 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fwrn\" (UniqueName: \"kubernetes.io/projected/b8e61afa-600c-4a01-a3ab-77b4feb567dd-kube-api-access-5fwrn\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:38.692297 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.692200 2575 generic.go:356] "Generic (PLEG): container finished" podID="b8e61afa-600c-4a01-a3ab-77b4feb567dd" containerID="397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc" exitCode=0 Jul 09 14:24:38.692297 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.692242 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" Jul 09 14:24:38.692297 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.692279 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" event={"ID":"b8e61afa-600c-4a01-a3ab-77b4feb567dd","Type":"ContainerDied","Data":"397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc"} Jul 09 14:24:38.692548 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.692334 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z" event={"ID":"b8e61afa-600c-4a01-a3ab-77b4feb567dd","Type":"ContainerDied","Data":"2a33062b61f74f29072fcc00db3a732ddcecf029251d06523b20c00b583a148e"} Jul 09 14:24:38.692548 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.692357 2575 scope.go:117] "RemoveContainer" containerID="397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc" Jul 09 14:24:38.702013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.701993 2575 scope.go:117] "RemoveContainer" containerID="397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc" Jul 09 14:24:38.702323 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:24:38.702303 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc\": container with ID starting with 397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc not found: ID does not exist" containerID="397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc" Jul 09 14:24:38.702376 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.702339 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc"} err="failed to get container status \"397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc\": rpc error: code = NotFound desc = could not find container \"397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc\": container with ID starting with 397e1c229e0df6e74c58e7987d546c1d93f287412367ee6478fbbdd2f5c51bdc not found: ID does not exist" Jul 09 14:24:38.715014 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.714984 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z"] Jul 09 14:24:38.726085 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:38.726051 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f4ec3dfc-openshift-default-75c86779c9-x2w6z"] Jul 09 14:24:40.638319 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:40.638281 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8e61afa-600c-4a01-a3ab-77b4feb567dd" path="/var/lib/kubelet/pods/b8e61afa-600c-4a01-a3ab-77b4feb567dd/volumes" Jul 09 14:24:43.480550 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.480518 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb"] Jul 09 14:24:43.480967 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.480953 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8e61afa-600c-4a01-a3ab-77b4feb567dd" containerName="istio-proxy" Jul 09 14:24:43.480967 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.480968 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8e61afa-600c-4a01-a3ab-77b4feb567dd" containerName="istio-proxy" Jul 09 14:24:43.481053 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.481044 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8e61afa-600c-4a01-a3ab-77b4feb567dd" containerName="istio-proxy" Jul 09 14:24:43.484438 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.484418 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.489812 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.489791 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6c569375-openshift-default-dockercfg-b75k4\"" Jul 09 14:24:43.498341 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.498319 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb"] Jul 09 14:24:43.555415 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.555378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrtqw\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-kube-api-access-hrtqw\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.555594 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.555422 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-certs\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.555594 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.555482 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-credential-socket\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.555594 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.555527 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-data\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.555709 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.555597 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-envoy\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.555709 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.555627 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-token\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.555709 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.555647 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-podinfo\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.555709 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.555691 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istiod-ca-cert\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.555709 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.555707 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-socket\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.656703 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.656665 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-token\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.656703 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.656704 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-podinfo\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.656731 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istiod-ca-cert\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.656783 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-socket\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.656839 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrtqw\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-kube-api-access-hrtqw\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.656911 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-certs\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.656956 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-credential-socket\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657020 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.657004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-data\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657353 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.657083 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-envoy\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657353 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.657233 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-socket\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657465 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.657422 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-credential-socket\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657518 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.657475 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istiod-ca-cert\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657579 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.657517 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-certs\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.657579 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.657565 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-data\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.659463 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.659438 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-envoy\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.659556 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.659443 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-podinfo\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.669826 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.669802 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrtqw\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-kube-api-access-hrtqw\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.670185 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.670164 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-token\") pod \"e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.796196 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.796157 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:43.946406 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.946377 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb"] Jul 09 14:24:43.947925 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:24:43.947895 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d930df2_b60f_4647_bd7a_bd3d46a866a3.slice/crio-eea8e052b5530ed165355efb4e7ea681e2362c0709f2b4660ed9661dd722b0dd WatchSource:0}: Error finding container eea8e052b5530ed165355efb4e7ea681e2362c0709f2b4660ed9661dd722b0dd: Status 404 returned error can't find the container with id eea8e052b5530ed165355efb4e7ea681e2362c0709f2b4660ed9661dd722b0dd Jul 09 14:24:43.950047 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.950015 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:24:43.950116 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.950086 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:24:43.950161 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:43.950118 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:24:44.738425 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:44.738373 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" event={"ID":"3d930df2-b60f-4647-bd7a-bd3d46a866a3","Type":"ContainerStarted","Data":"619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f"} Jul 09 14:24:44.738971 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:44.738496 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" event={"ID":"3d930df2-b60f-4647-bd7a-bd3d46a866a3","Type":"ContainerStarted","Data":"eea8e052b5530ed165355efb4e7ea681e2362c0709f2b4660ed9661dd722b0dd"} Jul 09 14:24:44.773666 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:44.773586 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" podStartSLOduration=1.773567374 podStartE2EDuration="1.773567374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:24:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:24:44.772532988 +0000 UTC m=+1748.626484469" watchObservedRunningTime="2026-07-09 14:24:44.773567374 +0000 UTC m=+1748.627518851" Jul 09 14:24:44.796515 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:44.796458 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:45.802377 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:45.802338 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:46.749874 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:46.749830 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:46.750962 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:46.750936 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:49.980165 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:49.980120 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb"] Jul 09 14:24:49.980647 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:49.980411 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" podUID="3d930df2-b60f-4647-bd7a-bd3d46a866a3" containerName="istio-proxy" containerID="cri-o://619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f" gracePeriod=30 Jul 09 14:24:55.242064 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.241996 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:55.379123 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379086 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-envoy\") pod \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " Jul 09 14:24:55.379298 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379134 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-socket\") pod \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " Jul 09 14:24:55.379298 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379157 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-certs\") pod \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " Jul 09 14:24:55.379298 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379189 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-token\") pod \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " Jul 09 14:24:55.379298 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379238 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-credential-socket\") pod \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " Jul 09 14:24:55.379298 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379274 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istiod-ca-cert\") pod \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " Jul 09 14:24:55.379540 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379344 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-podinfo\") pod \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " Jul 09 14:24:55.379540 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379373 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-data\") pod \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " Jul 09 14:24:55.379540 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379401 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrtqw\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-kube-api-access-hrtqw\") pod \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\" (UID: \"3d930df2-b60f-4647-bd7a-bd3d46a866a3\") " Jul 09 14:24:55.379540 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379449 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d930df2-b60f-4647-bd7a-bd3d46a866a3" (UID: "3d930df2-b60f-4647-bd7a-bd3d46a866a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:55.379755 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379549 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d930df2-b60f-4647-bd7a-bd3d46a866a3" (UID: "3d930df2-b60f-4647-bd7a-bd3d46a866a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:55.379755 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379718 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d930df2-b60f-4647-bd7a-bd3d46a866a3" (UID: "3d930df2-b60f-4647-bd7a-bd3d46a866a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:55.379887 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379749 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d930df2-b60f-4647-bd7a-bd3d46a866a3" (UID: "3d930df2-b60f-4647-bd7a-bd3d46a866a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:24:55.379887 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379772 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d930df2-b60f-4647-bd7a-bd3d46a866a3" (UID: "3d930df2-b60f-4647-bd7a-bd3d46a866a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:55.379887 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379816 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:55.379887 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.379877 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:55.381661 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.381631 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-kube-api-access-hrtqw" (OuterVolumeSpecName: "kube-api-access-hrtqw") pod "3d930df2-b60f-4647-bd7a-bd3d46a866a3" (UID: "3d930df2-b60f-4647-bd7a-bd3d46a866a3"). InnerVolumeSpecName "kube-api-access-hrtqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:24:55.381781 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.381631 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d930df2-b60f-4647-bd7a-bd3d46a866a3" (UID: "3d930df2-b60f-4647-bd7a-bd3d46a866a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:24:55.382049 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.382023 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d930df2-b60f-4647-bd7a-bd3d46a866a3" (UID: "3d930df2-b60f-4647-bd7a-bd3d46a866a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:24:55.382049 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.382035 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d930df2-b60f-4647-bd7a-bd3d46a866a3" (UID: "3d930df2-b60f-4647-bd7a-bd3d46a866a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:24:55.480965 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.480922 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:55.480965 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.480951 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:55.480965 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.480961 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:55.480965 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.480969 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:55.480965 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.480976 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:55.481265 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.480984 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d930df2-b60f-4647-bd7a-bd3d46a866a3-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:55.481265 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.480992 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrtqw\" (UniqueName: \"kubernetes.io/projected/3d930df2-b60f-4647-bd7a-bd3d46a866a3-kube-api-access-hrtqw\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:24:55.802437 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.802397 2575 generic.go:356] "Generic (PLEG): container finished" podID="3d930df2-b60f-4647-bd7a-bd3d46a866a3" containerID="619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f" exitCode=0 Jul 09 14:24:55.802617 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.802472 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" Jul 09 14:24:55.802617 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.802470 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" event={"ID":"3d930df2-b60f-4647-bd7a-bd3d46a866a3","Type":"ContainerDied","Data":"619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f"} Jul 09 14:24:55.802617 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.802601 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb" event={"ID":"3d930df2-b60f-4647-bd7a-bd3d46a866a3","Type":"ContainerDied","Data":"eea8e052b5530ed165355efb4e7ea681e2362c0709f2b4660ed9661dd722b0dd"} Jul 09 14:24:55.802730 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.802629 2575 scope.go:117] "RemoveContainer" containerID="619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f" Jul 09 14:24:55.813296 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.813272 2575 scope.go:117] "RemoveContainer" containerID="619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f" Jul 09 14:24:55.813613 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:24:55.813593 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f\": container with ID starting with 619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f not found: ID does not exist" containerID="619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f" Jul 09 14:24:55.813688 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.813622 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f"} err="failed to get container status \"619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f\": rpc error: code = NotFound desc = could not find container \"619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f\": container with ID starting with 619b7a67e7e4970c7e637c0c6350b7a3fc1a4e4e12d03432e4fbdadf35d65f3f not found: ID does not exist" Jul 09 14:24:55.826144 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.826109 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb"] Jul 09 14:24:55.831578 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:55.831551 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6c569375-openshift-default-66c85f98c5-bxxxb"] Jul 09 14:24:56.638987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:24:56.638954 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d930df2-b60f-4647-bd7a-bd3d46a866a3" path="/var/lib/kubelet/pods/3d930df2-b60f-4647-bd7a-bd3d46a866a3/volumes" Jul 09 14:26:35.098831 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.098786 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2"] Jul 09 14:26:35.099461 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.099422 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d930df2-b60f-4647-bd7a-bd3d46a866a3" containerName="istio-proxy" Jul 09 14:26:35.099461 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.099443 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d930df2-b60f-4647-bd7a-bd3d46a866a3" containerName="istio-proxy" Jul 09 14:26:35.099603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.099587 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d930df2-b60f-4647-bd7a-bd3d46a866a3" containerName="istio-proxy" Jul 09 14:26:35.101820 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.101799 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.105315 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.105282 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9c958fc8-openshift-default-dockercfg-2w8lt\"" Jul 09 14:26:35.135921 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.135885 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2"] Jul 09 14:26:35.221348 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.221285 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggwkg\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-kube-api-access-ggwkg\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.221348 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.221358 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23274bf6-170b-4386-bd59-ac145b1c904b-istiod-ca-cert\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.221607 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.221394 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-istio-token\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.221607 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.221505 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-credential-socket\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.221607 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.221544 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-certs\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.221607 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.221566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-data\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.221787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.221651 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-socket\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.221787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.221673 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23274bf6-170b-4386-bd59-ac145b1c904b-istio-podinfo\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.221787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.221705 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-envoy\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.322361 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322330 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-credential-socket\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.322361 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322365 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-certs\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.322620 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322391 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-data\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.322620 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322427 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-socket\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.322620 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322442 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23274bf6-170b-4386-bd59-ac145b1c904b-istio-podinfo\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.322620 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322467 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-envoy\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.322620 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322488 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggwkg\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-kube-api-access-ggwkg\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.322620 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322516 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23274bf6-170b-4386-bd59-ac145b1c904b-istiod-ca-cert\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.322620 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322548 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-istio-token\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.323062 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322795 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-certs\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.323062 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.322820 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-credential-socket\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.323175 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.323091 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-socket\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.323238 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.323170 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-data\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.323449 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.323426 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23274bf6-170b-4386-bd59-ac145b1c904b-istiod-ca-cert\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.325030 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.325012 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-envoy\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.325096 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.325083 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23274bf6-170b-4386-bd59-ac145b1c904b-istio-podinfo\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.331905 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.331886 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-istio-token\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.332394 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.332371 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggwkg\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-kube-api-access-ggwkg\") pod \"e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.414369 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.414275 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:35.588244 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.588206 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2"] Jul 09 14:26:35.590037 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:26:35.590010 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23274bf6_170b_4386_bd59_ac145b1c904b.slice/crio-73f0b96214c2581f48f3fe08e3623ac54b6c0bafcaca952d9942419a20e61d15 WatchSource:0}: Error finding container 73f0b96214c2581f48f3fe08e3623ac54b6c0bafcaca952d9942419a20e61d15: Status 404 returned error can't find the container with id 73f0b96214c2581f48f3fe08e3623ac54b6c0bafcaca952d9942419a20e61d15 Jul 09 14:26:35.592091 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.592057 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:26:35.592193 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.592123 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:26:35.592193 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:35.592169 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:26:36.390364 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:36.390319 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" event={"ID":"23274bf6-170b-4386-bd59-ac145b1c904b","Type":"ContainerStarted","Data":"ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78"} Jul 09 14:26:36.390943 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:36.390385 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" event={"ID":"23274bf6-170b-4386-bd59-ac145b1c904b","Type":"ContainerStarted","Data":"73f0b96214c2581f48f3fe08e3623ac54b6c0bafcaca952d9942419a20e61d15"} Jul 09 14:26:36.415013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:36.414980 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:36.420062 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:36.420027 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:36.420292 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:36.420012 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" podStartSLOduration=1.4199955 podStartE2EDuration="1.4199955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:26:36.419910115 +0000 UTC m=+1860.273861592" watchObservedRunningTime="2026-07-09 14:26:36.4199955 +0000 UTC m=+1860.273946978" Jul 09 14:26:37.396494 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:37.396456 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:37.397462 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:37.397440 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:52.398614 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:52.398572 2575 patch_prober.go:28] interesting pod/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 09 14:26:52.399126 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:52.398637 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 09 14:26:52.484702 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:52.484666 2575 generic.go:356] "Generic (PLEG): container finished" podID="23274bf6-170b-4386-bd59-ac145b1c904b" containerID="ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78" exitCode=0 Jul 09 14:26:52.484914 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:52.484741 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" event={"ID":"23274bf6-170b-4386-bd59-ac145b1c904b","Type":"ContainerDied","Data":"ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78"} Jul 09 14:26:52.485219 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:52.485204 2575 scope.go:117] "RemoveContainer" containerID="ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78" Jul 09 14:26:52.486324 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:52.486296 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:26:52.486398 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:52.486366 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:26:52.486437 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:52.486402 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:26:53.502959 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:53.502917 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" event={"ID":"23274bf6-170b-4386-bd59-ac145b1c904b","Type":"ContainerStarted","Data":"8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545"} Jul 09 14:26:54.415419 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:54.415386 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:26:54.415676 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:54.415648 2575 patch_prober.go:28] interesting pod/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 09 14:26:54.415766 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:54.415714 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 09 14:26:55.415275 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:55.415235 2575 patch_prober.go:28] interesting pod/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 09 14:26:55.415703 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:55.415299 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 09 14:26:56.415413 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:56.415370 2575 patch_prober.go:28] interesting pod/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 09 14:26:56.415835 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:56.415435 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 09 14:26:57.415616 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:57.415571 2575 patch_prober.go:28] interesting pod/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 09 14:26:57.416145 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:57.415643 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 09 14:26:57.647964 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:57.647930 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2"] Jul 09 14:26:57.648207 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:26:57.648164 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" containerID="cri-o://8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545" gracePeriod=30 Jul 09 14:27:02.913510 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:02.913445 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:27:03.008246 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008211 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-data\") pod \"23274bf6-170b-4386-bd59-ac145b1c904b\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " Jul 09 14:27:03.008246 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008253 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-istio-token\") pod \"23274bf6-170b-4386-bd59-ac145b1c904b\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " Jul 09 14:27:03.008519 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008278 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-certs\") pod \"23274bf6-170b-4386-bd59-ac145b1c904b\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " Jul 09 14:27:03.008519 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008371 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23274bf6-170b-4386-bd59-ac145b1c904b-istiod-ca-cert\") pod \"23274bf6-170b-4386-bd59-ac145b1c904b\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " Jul 09 14:27:03.008519 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008404 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-envoy\") pod \"23274bf6-170b-4386-bd59-ac145b1c904b\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " Jul 09 14:27:03.008519 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008434 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggwkg\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-kube-api-access-ggwkg\") pod \"23274bf6-170b-4386-bd59-ac145b1c904b\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " Jul 09 14:27:03.008519 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008459 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-credential-socket\") pod \"23274bf6-170b-4386-bd59-ac145b1c904b\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " Jul 09 14:27:03.008519 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008494 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23274bf6-170b-4386-bd59-ac145b1c904b-istio-podinfo\") pod \"23274bf6-170b-4386-bd59-ac145b1c904b\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " Jul 09 14:27:03.008880 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008553 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-socket\") pod \"23274bf6-170b-4386-bd59-ac145b1c904b\" (UID: \"23274bf6-170b-4386-bd59-ac145b1c904b\") " Jul 09 14:27:03.008880 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008579 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23274bf6-170b-4386-bd59-ac145b1c904b" (UID: "23274bf6-170b-4386-bd59-ac145b1c904b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:27:03.009007 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.008883 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23274bf6-170b-4386-bd59-ac145b1c904b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23274bf6-170b-4386-bd59-ac145b1c904b" (UID: "23274bf6-170b-4386-bd59-ac145b1c904b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:27:03.009109 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.009085 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:27:03.009167 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.009122 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23274bf6-170b-4386-bd59-ac145b1c904b-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:27:03.009167 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.009081 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23274bf6-170b-4386-bd59-ac145b1c904b" (UID: "23274bf6-170b-4386-bd59-ac145b1c904b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:27:03.009269 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.009218 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23274bf6-170b-4386-bd59-ac145b1c904b" (UID: "23274bf6-170b-4386-bd59-ac145b1c904b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:27:03.009269 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.009241 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-data" (OuterVolumeSpecName: "istio-data") pod "23274bf6-170b-4386-bd59-ac145b1c904b" (UID: "23274bf6-170b-4386-bd59-ac145b1c904b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:27:03.010801 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.010777 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23274bf6-170b-4386-bd59-ac145b1c904b" (UID: "23274bf6-170b-4386-bd59-ac145b1c904b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:27:03.010950 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.010900 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-istio-token" (OuterVolumeSpecName: "istio-token") pod "23274bf6-170b-4386-bd59-ac145b1c904b" (UID: "23274bf6-170b-4386-bd59-ac145b1c904b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:27:03.011319 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.011299 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23274bf6-170b-4386-bd59-ac145b1c904b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23274bf6-170b-4386-bd59-ac145b1c904b" (UID: "23274bf6-170b-4386-bd59-ac145b1c904b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:27:03.011390 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.011323 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-kube-api-access-ggwkg" (OuterVolumeSpecName: "kube-api-access-ggwkg") pod "23274bf6-170b-4386-bd59-ac145b1c904b" (UID: "23274bf6-170b-4386-bd59-ac145b1c904b"). InnerVolumeSpecName "kube-api-access-ggwkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:27:03.110694 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.110641 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23274bf6-170b-4386-bd59-ac145b1c904b-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:27:03.110694 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.110689 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:27:03.110694 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.110698 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:27:03.110694 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.110707 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:27:03.110694 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.110714 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:27:03.111056 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.110721 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggwkg\" (UniqueName: \"kubernetes.io/projected/23274bf6-170b-4386-bd59-ac145b1c904b-kube-api-access-ggwkg\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:27:03.111056 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.110731 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23274bf6-170b-4386-bd59-ac145b1c904b-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:27:03.565030 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.564997 2575 generic.go:356] "Generic (PLEG): container finished" podID="23274bf6-170b-4386-bd59-ac145b1c904b" containerID="8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545" exitCode=0 Jul 09 14:27:03.565204 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.565048 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" Jul 09 14:27:03.565204 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.565085 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" event={"ID":"23274bf6-170b-4386-bd59-ac145b1c904b","Type":"ContainerDied","Data":"8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545"} Jul 09 14:27:03.565204 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.565142 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2" event={"ID":"23274bf6-170b-4386-bd59-ac145b1c904b","Type":"ContainerDied","Data":"73f0b96214c2581f48f3fe08e3623ac54b6c0bafcaca952d9942419a20e61d15"} Jul 09 14:27:03.565204 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.565165 2575 scope.go:117] "RemoveContainer" containerID="8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545" Jul 09 14:27:03.575821 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.575801 2575 scope.go:117] "RemoveContainer" containerID="ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78" Jul 09 14:27:03.584809 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.584787 2575 scope.go:117] "RemoveContainer" containerID="8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545" Jul 09 14:27:03.585117 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:27:03.585096 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545\": container with ID starting with 8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545 not found: ID does not exist" containerID="8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545" Jul 09 14:27:03.585185 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.585126 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545"} err="failed to get container status \"8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545\": rpc error: code = NotFound desc = could not find container \"8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545\": container with ID starting with 8174d3e4217dd921da325e67597d1b788ba8f9de2603d9d13e561b033aeb9545 not found: ID does not exist" Jul 09 14:27:03.585185 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.585155 2575 scope.go:117] "RemoveContainer" containerID="ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78" Jul 09 14:27:03.585435 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:27:03.585415 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78\": container with ID starting with ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78 not found: ID does not exist" containerID="ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78" Jul 09 14:27:03.585499 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.585447 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78"} err="failed to get container status \"ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78\": rpc error: code = NotFound desc = could not find container \"ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78\": container with ID starting with ad81cb89a33a8a543e984122a42063fbd8dc5e9e9494077c83c8600f65ee4f78 not found: ID does not exist" Jul 09 14:27:03.589127 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.589102 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2"] Jul 09 14:27:03.597279 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:03.597240 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9c958fc8-openshift-default-76cb558795-sjzn2"] Jul 09 14:27:04.638554 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:04.638522 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" path="/var/lib/kubelet/pods/23274bf6-170b-4386-bd59-ac145b1c904b/volumes" Jul 09 14:27:17.767495 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.767453 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn"] Jul 09 14:27:17.768215 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.768188 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" Jul 09 14:27:17.768302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.768219 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" Jul 09 14:27:17.768302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.768248 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" Jul 09 14:27:17.768302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.768257 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" Jul 09 14:27:17.768453 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.768374 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" Jul 09 14:27:17.771596 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.771574 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.775150 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.775128 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bea9ff30-openshift-default-dockercfg-p44q4\"" Jul 09 14:27:17.794829 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.794794 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn"] Jul 09 14:27:17.836071 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.836024 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-socket\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.836423 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.836377 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istiod-ca-cert\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.836576 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.836559 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2mm6\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-kube-api-access-q2mm6\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.836697 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.836683 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-token\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.836795 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.836782 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-credential-socket\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.836953 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.836934 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-envoy\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.837139 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.837116 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-data\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.837277 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.837150 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-certs\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.837277 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.837192 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-podinfo\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.937809 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.937760 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-socket\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938000 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.937893 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istiod-ca-cert\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938000 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.937947 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2mm6\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-kube-api-access-q2mm6\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938000 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.937987 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-token\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938170 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.938018 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-credential-socket\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938314 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.938291 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-socket\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938416 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.938395 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-credential-socket\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938466 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.938443 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-envoy\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938564 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.938541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-data\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938605 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.938572 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-certs\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938647 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.938599 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istiod-ca-cert\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938647 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.938614 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-podinfo\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.938832 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.938812 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-data\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.939092 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.939064 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-certs\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.940560 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.940541 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-envoy\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.941470 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.941448 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-podinfo\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.958343 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.958316 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-token\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:17.958559 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:17.958533 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2mm6\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-kube-api-access-q2mm6\") pod \"e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:18.085066 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:18.084971 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:18.247546 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:18.247515 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn"] Jul 09 14:27:18.248927 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:27:18.248895 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0662dbba_0f01_4e25_9dfb_d4f7dd8bae89.slice/crio-38843ff51923b671e642095ac2c143ce2ccce132c1f77ad5dcfc125f2a181d96 WatchSource:0}: Error finding container 38843ff51923b671e642095ac2c143ce2ccce132c1f77ad5dcfc125f2a181d96: Status 404 returned error can't find the container with id 38843ff51923b671e642095ac2c143ce2ccce132c1f77ad5dcfc125f2a181d96 Jul 09 14:27:18.251691 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:18.251642 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:27:18.251817 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:18.251745 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:27:18.251817 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:18.251792 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:27:18.657300 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:18.657257 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" event={"ID":"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89","Type":"ContainerStarted","Data":"3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176"} Jul 09 14:27:18.657505 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:18.657317 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" event={"ID":"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89","Type":"ContainerStarted","Data":"38843ff51923b671e642095ac2c143ce2ccce132c1f77ad5dcfc125f2a181d96"} Jul 09 14:27:18.683277 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:18.683207 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" podStartSLOduration=1.683183055 podStartE2EDuration="1.683183055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:27:18.680896454 +0000 UTC m=+1902.534847931" watchObservedRunningTime="2026-07-09 14:27:18.683183055 +0000 UTC m=+1902.537134532" Jul 09 14:27:19.085808 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:19.085770 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:19.091291 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:19.091263 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:19.664540 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:19.664506 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:19.665428 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:19.665409 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:33.750889 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:33.750838 2575 generic.go:356] "Generic (PLEG): container finished" podID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerID="3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176" exitCode=0 Jul 09 14:27:33.750889 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:33.750878 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" event={"ID":"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89","Type":"ContainerDied","Data":"3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176"} Jul 09 14:27:33.751361 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:33.751280 2575 scope.go:117] "RemoveContainer" containerID="3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176" Jul 09 14:27:33.753744 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:33.753593 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:27:33.753819 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:33.753773 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:27:33.756051 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:33.753890 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:27:34.665033 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:34.664980 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:34.758917 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:34.758880 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" event={"ID":"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89","Type":"ContainerStarted","Data":"31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a"} Jul 09 14:27:35.085921 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:35.085883 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:35.087232 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:35.087200 2575 patch_prober.go:28] interesting pod/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 09 14:27:35.087517 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:35.087472 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 09 14:27:36.085896 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:36.085834 2575 patch_prober.go:28] interesting pod/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 09 14:27:36.086531 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:36.086494 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 09 14:27:37.085670 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:37.085572 2575 patch_prober.go:28] interesting pod/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 09 14:27:37.085980 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:37.085948 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 09 14:27:38.086054 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:38.086007 2575 patch_prober.go:28] interesting pod/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 09 14:27:38.086562 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:38.086087 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 09 14:27:39.090526 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:39.090492 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:39.090934 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:39.090799 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:27:39.091762 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:27:39.091746 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:29:02.388813 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:02.388776 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn"] Jul 09 14:29:02.389334 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:02.389040 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" containerID="cri-o://31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a" gracePeriod=30 Jul 09 14:29:07.647126 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.647097 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:29:07.754332 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754233 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-certs\") pod \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " Jul 09 14:29:07.754332 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754270 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-data\") pod \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " Jul 09 14:29:07.754332 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754291 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2mm6\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-kube-api-access-q2mm6\") pod \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " Jul 09 14:29:07.754626 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754346 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istiod-ca-cert\") pod \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " Jul 09 14:29:07.754626 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754372 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-podinfo\") pod \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " Jul 09 14:29:07.754626 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754405 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-socket\") pod \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " Jul 09 14:29:07.754626 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754443 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-envoy\") pod \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " Jul 09 14:29:07.754626 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754467 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-token\") pod \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " Jul 09 14:29:07.754626 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754507 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-credential-socket\") pod \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\" (UID: \"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89\") " Jul 09 14:29:07.755013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754722 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" (UID: "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:29:07.755013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754806 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" (UID: "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:29:07.755013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754933 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" (UID: "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:29:07.755013 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.754976 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" (UID: "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:29:07.755226 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.755025 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:29:07.755226 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.755050 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:29:07.755226 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.755065 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:29:07.755226 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.755206 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-data" (OuterVolumeSpecName: "istio-data") pod "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" (UID: "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:29:07.756987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.756954 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" (UID: "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:29:07.757260 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.757110 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" (UID: "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:29:07.757556 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.757536 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-kube-api-access-q2mm6" (OuterVolumeSpecName: "kube-api-access-q2mm6") pod "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" (UID: "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89"). InnerVolumeSpecName "kube-api-access-q2mm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:29:07.757623 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.757554 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-token" (OuterVolumeSpecName: "istio-token") pod "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" (UID: "0662dbba-0f01-4e25-9dfb-d4f7dd8bae89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:29:07.855594 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.855547 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:29:07.855594 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.855587 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2mm6\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-kube-api-access-q2mm6\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:29:07.855594 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.855598 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:29:07.855594 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.855607 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:29:07.855931 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.855615 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:29:07.855931 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:07.855623 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:29:08.320437 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.320403 2575 generic.go:356] "Generic (PLEG): container finished" podID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerID="31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a" exitCode=0 Jul 09 14:29:08.320657 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.320466 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" Jul 09 14:29:08.320657 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.320488 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" event={"ID":"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89","Type":"ContainerDied","Data":"31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a"} Jul 09 14:29:08.320657 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.320548 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn" event={"ID":"0662dbba-0f01-4e25-9dfb-d4f7dd8bae89","Type":"ContainerDied","Data":"38843ff51923b671e642095ac2c143ce2ccce132c1f77ad5dcfc125f2a181d96"} Jul 09 14:29:08.320657 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.320570 2575 scope.go:117] "RemoveContainer" containerID="31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a" Jul 09 14:29:08.331312 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.331276 2575 scope.go:117] "RemoveContainer" containerID="3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176" Jul 09 14:29:08.340739 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.340707 2575 scope.go:117] "RemoveContainer" containerID="31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a" Jul 09 14:29:08.341141 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:29:08.341090 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a\": container with ID starting with 31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a not found: ID does not exist" containerID="31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a" Jul 09 14:29:08.341141 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.341124 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a"} err="failed to get container status \"31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a\": rpc error: code = NotFound desc = could not find container \"31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a\": container with ID starting with 31f86b3b93fc5af0a05285f9ba84e45345ea144a76ffb410483d06c5cd7d4d4a not found: ID does not exist" Jul 09 14:29:08.341141 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.341146 2575 scope.go:117] "RemoveContainer" containerID="3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176" Jul 09 14:29:08.341603 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:29:08.341538 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176\": container with ID starting with 3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176 not found: ID does not exist" containerID="3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176" Jul 09 14:29:08.341603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.341570 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176"} err="failed to get container status \"3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176\": rpc error: code = NotFound desc = could not find container \"3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176\": container with ID starting with 3daf1dfc925a085f160cb9b0929d57651e2f0dfd9ebd11617c16a40c7c6c0176 not found: ID does not exist" Jul 09 14:29:08.350379 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.350339 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn"] Jul 09 14:29:08.352702 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.352667 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bea9ff30-openshift-default-8759889dc-9dtgn"] Jul 09 14:29:08.639006 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:08.638918 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" path="/var/lib/kubelet/pods/0662dbba-0f01-4e25-9dfb-d4f7dd8bae89/volumes" Jul 09 14:29:58.787024 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.786935 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n"] Jul 09 14:29:58.787537 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.787345 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" Jul 09 14:29:58.787537 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.787356 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" Jul 09 14:29:58.787537 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.787367 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" Jul 09 14:29:58.787537 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.787372 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" Jul 09 14:29:58.787537 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.787443 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="23274bf6-170b-4386-bd59-ac145b1c904b" containerName="istio-proxy" Jul 09 14:29:58.787537 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.787451 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" Jul 09 14:29:58.787537 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.787460 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0662dbba-0f01-4e25-9dfb-d4f7dd8bae89" containerName="istio-proxy" Jul 09 14:29:58.790820 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.790799 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:58.793741 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.793720 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-738843-openshift-default-dockercfg-k6glh\"" Jul 09 14:29:58.817199 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.817159 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n"] Jul 09 14:29:58.924793 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.924739 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-socket\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:58.925023 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.924814 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc791e24-f92b-4591-a63d-420cfdb94856-istiod-ca-cert\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:58.925023 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.924879 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-data\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:58.925023 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.924921 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-istio-token\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:58.925023 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.924994 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-credential-socket\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:58.925231 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.925036 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc791e24-f92b-4591-a63d-420cfdb94856-istio-podinfo\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:58.925231 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.925059 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-certs\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:58.925231 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.925183 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndmdl\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-kube-api-access-ndmdl\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:58.925368 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:58.925241 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-envoy\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.025938 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.025897 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-envoy\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026168 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.025974 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-socket\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026168 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026003 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc791e24-f92b-4591-a63d-420cfdb94856-istiod-ca-cert\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026168 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026027 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-data\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026168 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026052 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-istio-token\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026168 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026085 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-credential-socket\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026168 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026114 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc791e24-f92b-4591-a63d-420cfdb94856-istio-podinfo\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026168 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026141 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-certs\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026557 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026184 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndmdl\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-kube-api-access-ndmdl\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026557 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026482 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-socket\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026635 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026557 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-credential-socket\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026635 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026611 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-certs\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.026736 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.026715 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-data\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.027258 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.027234 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc791e24-f92b-4591-a63d-420cfdb94856-istiod-ca-cert\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.028563 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.028542 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-envoy\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.028788 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.028768 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc791e24-f92b-4591-a63d-420cfdb94856-istio-podinfo\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.035787 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.035763 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndmdl\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-kube-api-access-ndmdl\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.037415 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.037352 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-istio-token\") pod \"e2e-inf-a-738843-openshift-default-859846c547-hkr9n\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.102760 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.102712 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:29:59.252459 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.252428 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n"] Jul 09 14:29:59.253723 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:29:59.253700 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc791e24_f92b_4591_a63d_420cfdb94856.slice/crio-17406034a7be8fc71aece8f17badfc20de1201c5afa78c326db78909ff0ace54 WatchSource:0}: Error finding container 17406034a7be8fc71aece8f17badfc20de1201c5afa78c326db78909ff0ace54: Status 404 returned error can't find the container with id 17406034a7be8fc71aece8f17badfc20de1201c5afa78c326db78909ff0ace54 Jul 09 14:29:59.256119 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.256085 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:29:59.256203 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.256152 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:29:59.256203 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.256181 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:29:59.627140 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.627040 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" event={"ID":"fc791e24-f92b-4591-a63d-420cfdb94856","Type":"ContainerStarted","Data":"9ac2b6309684bfce95ea7a74a5cfe006f59f3e8f1ace4b4ebc243464e9a8577f"} Jul 09 14:29:59.627140 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.627104 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" event={"ID":"fc791e24-f92b-4591-a63d-420cfdb94856","Type":"ContainerStarted","Data":"17406034a7be8fc71aece8f17badfc20de1201c5afa78c326db78909ff0ace54"} Jul 09 14:29:59.649822 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:29:59.649767 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" podStartSLOduration=1.649748379 podStartE2EDuration="1.649748379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:59.647634876 +0000 UTC m=+2063.501586355" watchObservedRunningTime="2026-07-09 14:29:59.649748379 +0000 UTC m=+2063.503699864" Jul 09 14:30:00.103683 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:00.103643 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:30:00.108934 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:00.108910 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:30:00.638243 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:00.638207 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:30:00.638411 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:00.638287 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:30:07.311526 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.311490 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8"] Jul 09 14:30:07.315366 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.315344 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.318057 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.318036 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f1a2e7-openshift-default-dockercfg-dtwlq\"" Jul 09 14:30:07.326668 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.326633 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8"] Jul 09 14:30:07.403608 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.403563 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svr5f\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-kube-api-access-svr5f\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.403805 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.403614 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istiod-ca-cert\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.403805 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.403704 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-podinfo\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.403937 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.403795 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-certs\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.403937 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.403859 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-socket\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.403937 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.403896 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-token\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.404092 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.403955 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-data\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.404092 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.403986 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-envoy\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.404092 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.404010 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-credential-socket\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505017 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.504972 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-data\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505017 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-envoy\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505290 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505032 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-credential-socket\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505290 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svr5f\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-kube-api-access-svr5f\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505290 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505221 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istiod-ca-cert\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505459 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505310 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-podinfo\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505459 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505359 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-certs\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505459 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505384 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-data\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505459 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505398 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-socket\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505459 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-token\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505459 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505462 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-credential-socket\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.505777 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505647 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-socket\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.506047 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.505813 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-certs\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.506047 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.506014 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istiod-ca-cert\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.507841 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.507803 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-envoy\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.508416 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.508391 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-podinfo\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.513876 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.513818 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-token\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.513976 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.513884 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svr5f\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-kube-api-access-svr5f\") pod \"e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.628328 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.628237 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:07.791530 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:30:07.791497 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08b5ae8_8dce_4aec_ba2b_17e47d30e416.slice/crio-5e16a2b5ac52e379fcbde71d208411dabea385a8f87adfc6e00d4601d730f52b WatchSource:0}: Error finding container 5e16a2b5ac52e379fcbde71d208411dabea385a8f87adfc6e00d4601d730f52b: Status 404 returned error can't find the container with id 5e16a2b5ac52e379fcbde71d208411dabea385a8f87adfc6e00d4601d730f52b Jul 09 14:30:07.791828 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.791801 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8"] Jul 09 14:30:07.793991 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.793955 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:30:07.794088 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.794030 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:30:07.794088 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:07.794060 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:30:08.700880 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:08.700815 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" event={"ID":"d08b5ae8-8dce-4aec-ba2b-17e47d30e416","Type":"ContainerStarted","Data":"deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa"} Jul 09 14:30:08.701635 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:08.701589 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" event={"ID":"d08b5ae8-8dce-4aec-ba2b-17e47d30e416","Type":"ContainerStarted","Data":"5e16a2b5ac52e379fcbde71d208411dabea385a8f87adfc6e00d4601d730f52b"} Jul 09 14:30:08.725130 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:08.725076 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" podStartSLOduration=1.72505916 podStartE2EDuration="1.72505916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:30:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:30:08.723696043 +0000 UTC m=+2072.577647518" watchObservedRunningTime="2026-07-09 14:30:08.72505916 +0000 UTC m=+2072.579010636" Jul 09 14:30:09.628585 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:09.628543 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:09.633892 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:09.633864 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:09.707887 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:09.707833 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:09.708818 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:09.708797 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:15.636097 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:15.635996 2575 patch_prober.go:28] interesting pod/e2e-inf-a-738843-openshift-default-859846c547-hkr9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 14:30:15.636097 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:15.636075 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 14:30:15.745484 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:15.745397 2575 generic.go:356] "Generic (PLEG): container finished" podID="fc791e24-f92b-4591-a63d-420cfdb94856" containerID="9ac2b6309684bfce95ea7a74a5cfe006f59f3e8f1ace4b4ebc243464e9a8577f" exitCode=0 Jul 09 14:30:15.745484 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:15.745442 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" event={"ID":"fc791e24-f92b-4591-a63d-420cfdb94856","Type":"ContainerDied","Data":"9ac2b6309684bfce95ea7a74a5cfe006f59f3e8f1ace4b4ebc243464e9a8577f"} Jul 09 14:30:15.745929 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:15.745915 2575 scope.go:117] "RemoveContainer" containerID="9ac2b6309684bfce95ea7a74a5cfe006f59f3e8f1ace4b4ebc243464e9a8577f" Jul 09 14:30:15.746993 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:15.746962 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:30:15.747073 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:15.747038 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:30:15.747110 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:15.747072 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:30:16.754062 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:16.754024 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" event={"ID":"fc791e24-f92b-4591-a63d-420cfdb94856","Type":"ContainerStarted","Data":"6e577ef29bc7e6c3500d7b6fe1714fa44147a219395de564c965ce0cc20d00a2"} Jul 09 14:30:17.103815 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:17.103339 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:30:17.103815 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:17.103475 2575 patch_prober.go:28] interesting pod/e2e-inf-a-738843-openshift-default-859846c547-hkr9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 14:30:17.103815 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:17.103533 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 14:30:18.103989 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:18.103947 2575 patch_prober.go:28] interesting pod/e2e-inf-a-738843-openshift-default-859846c547-hkr9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 14:30:18.104632 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:18.104598 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 14:30:19.103759 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:19.103709 2575 patch_prober.go:28] interesting pod/e2e-inf-a-738843-openshift-default-859846c547-hkr9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 14:30:19.104010 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:19.103781 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 14:30:20.103414 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:20.103362 2575 patch_prober.go:28] interesting pod/e2e-inf-a-738843-openshift-default-859846c547-hkr9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 09 14:30:20.103639 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:20.103438 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 09 14:30:21.108424 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:21.108385 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:30:21.108813 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:21.108788 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:30:21.109629 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:21.109613 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:30:24.709596 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:24.709496 2575 patch_prober.go:28] interesting pod/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 14:30:24.709596 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:24.709562 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 14:30:24.809967 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:24.809935 2575 generic.go:356] "Generic (PLEG): container finished" podID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerID="deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa" exitCode=0 Jul 09 14:30:24.810158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:24.809985 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" event={"ID":"d08b5ae8-8dce-4aec-ba2b-17e47d30e416","Type":"ContainerDied","Data":"deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa"} Jul 09 14:30:24.810405 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:24.810386 2575 scope.go:117] "RemoveContainer" containerID="deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa" Jul 09 14:30:24.811468 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:24.811439 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:30:24.811554 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:24.811540 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:30:24.811596 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:24.811578 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:30:25.820576 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:25.820533 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" event={"ID":"d08b5ae8-8dce-4aec-ba2b-17e47d30e416","Type":"ContainerStarted","Data":"ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526"} Jul 09 14:30:26.628540 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:26.628503 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:26.628797 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:26.628775 2575 patch_prober.go:28] interesting pod/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 14:30:26.628926 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:26.628891 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 14:30:27.629246 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:27.629201 2575 patch_prober.go:28] interesting pod/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 14:30:27.629684 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:27.629275 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 14:30:28.629548 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:28.629512 2575 patch_prober.go:28] interesting pod/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 14:30:28.630014 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:28.629580 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 14:30:29.628839 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:29.628797 2575 patch_prober.go:28] interesting pod/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 09 14:30:29.629077 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:29.628912 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 09 14:30:30.638628 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:30.638599 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:30.639139 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:30.639124 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:30:30.640201 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:30:30.640180 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:32:04.213865 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:04.213812 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n"] Jul 09 14:32:04.214369 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:04.214060 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" containerID="cri-o://6e577ef29bc7e6c3500d7b6fe1714fa44147a219395de564c965ce0cc20d00a2" gracePeriod=30 Jul 09 14:32:06.111579 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:06.111537 2575 patch_prober.go:28] interesting pod/e2e-inf-a-738843-openshift-default-859846c547-hkr9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:32:06.112114 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:06.111720 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:32:09.443932 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.443897 2575 generic.go:356] "Generic (PLEG): container finished" podID="fc791e24-f92b-4591-a63d-420cfdb94856" containerID="6e577ef29bc7e6c3500d7b6fe1714fa44147a219395de564c965ce0cc20d00a2" exitCode=0 Jul 09 14:32:09.444365 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.443973 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" event={"ID":"fc791e24-f92b-4591-a63d-420cfdb94856","Type":"ContainerDied","Data":"6e577ef29bc7e6c3500d7b6fe1714fa44147a219395de564c965ce0cc20d00a2"} Jul 09 14:32:09.444365 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.444040 2575 scope.go:117] "RemoveContainer" containerID="9ac2b6309684bfce95ea7a74a5cfe006f59f3e8f1ace4b4ebc243464e9a8577f" Jul 09 14:32:09.486880 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.486789 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:32:09.537533 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537494 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-credential-socket\") pod \"fc791e24-f92b-4591-a63d-420cfdb94856\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " Jul 09 14:32:09.537733 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537552 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc791e24-f92b-4591-a63d-420cfdb94856-istiod-ca-cert\") pod \"fc791e24-f92b-4591-a63d-420cfdb94856\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " Jul 09 14:32:09.537733 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537576 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-socket\") pod \"fc791e24-f92b-4591-a63d-420cfdb94856\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " Jul 09 14:32:09.537733 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537620 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-istio-token\") pod \"fc791e24-f92b-4591-a63d-420cfdb94856\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " Jul 09 14:32:09.537733 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537651 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc791e24-f92b-4591-a63d-420cfdb94856-istio-podinfo\") pod \"fc791e24-f92b-4591-a63d-420cfdb94856\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " Jul 09 14:32:09.537733 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537676 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndmdl\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-kube-api-access-ndmdl\") pod \"fc791e24-f92b-4591-a63d-420cfdb94856\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " Jul 09 14:32:09.538033 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537747 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-data\") pod \"fc791e24-f92b-4591-a63d-420cfdb94856\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " Jul 09 14:32:09.538033 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537775 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-certs\") pod \"fc791e24-f92b-4591-a63d-420cfdb94856\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " Jul 09 14:32:09.538033 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537815 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-envoy\") pod \"fc791e24-f92b-4591-a63d-420cfdb94856\" (UID: \"fc791e24-f92b-4591-a63d-420cfdb94856\") " Jul 09 14:32:09.538033 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.537899 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc791e24-f92b-4591-a63d-420cfdb94856" (UID: "fc791e24-f92b-4591-a63d-420cfdb94856"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:09.538240 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.538030 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc791e24-f92b-4591-a63d-420cfdb94856" (UID: "fc791e24-f92b-4591-a63d-420cfdb94856"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:09.538240 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.538075 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc791e24-f92b-4591-a63d-420cfdb94856-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc791e24-f92b-4591-a63d-420cfdb94856" (UID: "fc791e24-f92b-4591-a63d-420cfdb94856"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:32:09.538366 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.538261 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc791e24-f92b-4591-a63d-420cfdb94856" (UID: "fc791e24-f92b-4591-a63d-420cfdb94856"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:09.539205 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.539174 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc791e24-f92b-4591-a63d-420cfdb94856" (UID: "fc791e24-f92b-4591-a63d-420cfdb94856"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:09.539340 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.539326 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc791e24-f92b-4591-a63d-420cfdb94856-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:09.539409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.539348 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:09.539409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.539362 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:09.539409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.539375 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:09.539409 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.539387 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:09.540330 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.540294 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-kube-api-access-ndmdl" (OuterVolumeSpecName: "kube-api-access-ndmdl") pod "fc791e24-f92b-4591-a63d-420cfdb94856" (UID: "fc791e24-f92b-4591-a63d-420cfdb94856"). InnerVolumeSpecName "kube-api-access-ndmdl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:32:09.540330 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.540294 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc791e24-f92b-4591-a63d-420cfdb94856" (UID: "fc791e24-f92b-4591-a63d-420cfdb94856"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:09.540514 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.540398 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc791e24-f92b-4591-a63d-420cfdb94856-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc791e24-f92b-4591-a63d-420cfdb94856" (UID: "fc791e24-f92b-4591-a63d-420cfdb94856"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:32:09.540574 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.540509 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc791e24-f92b-4591-a63d-420cfdb94856" (UID: "fc791e24-f92b-4591-a63d-420cfdb94856"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:32:09.640603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.640557 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc791e24-f92b-4591-a63d-420cfdb94856-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:09.640603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.640592 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:09.640603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.640604 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc791e24-f92b-4591-a63d-420cfdb94856-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:09.640603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:09.640612 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndmdl\" (UniqueName: \"kubernetes.io/projected/fc791e24-f92b-4591-a63d-420cfdb94856-kube-api-access-ndmdl\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:10.451968 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:10.451930 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" event={"ID":"fc791e24-f92b-4591-a63d-420cfdb94856","Type":"ContainerDied","Data":"17406034a7be8fc71aece8f17badfc20de1201c5afa78c326db78909ff0ace54"} Jul 09 14:32:10.451968 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:10.451971 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n" Jul 09 14:32:10.452488 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:10.451996 2575 scope.go:117] "RemoveContainer" containerID="6e577ef29bc7e6c3500d7b6fe1714fa44147a219395de564c965ce0cc20d00a2" Jul 09 14:32:10.478378 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:10.478336 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n"] Jul 09 14:32:10.484445 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:10.484411 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-738843-openshift-default-859846c547-hkr9n"] Jul 09 14:32:10.638357 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:10.638309 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" path="/var/lib/kubelet/pods/fc791e24-f92b-4591-a63d-420cfdb94856/volumes" Jul 09 14:32:11.432279 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:11.432237 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8"] Jul 09 14:32:11.432825 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:11.432771 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" containerID="cri-o://ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526" gracePeriod=30 Jul 09 14:32:12.367339 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.367299 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz"] Jul 09 14:32:12.367817 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.367801 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" Jul 09 14:32:12.367893 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.367821 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" Jul 09 14:32:12.367893 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.367829 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" Jul 09 14:32:12.367893 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.367835 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" Jul 09 14:32:12.367990 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.367935 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" Jul 09 14:32:12.367990 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.367947 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc791e24-f92b-4591-a63d-420cfdb94856" containerName="istio-proxy" Jul 09 14:32:12.371487 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.371465 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.375501 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.375479 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-38f05c-openshift-default-dockercfg-9xwmw\"" Jul 09 14:32:12.392888 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.392833 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz"] Jul 09 14:32:12.465192 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.465138 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4llpd\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-kube-api-access-4llpd\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.465412 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.465249 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-token\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.465412 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.465304 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istiod-ca-cert\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.465412 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.465335 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-credential-socket\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.465412 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.465389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-socket\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.465641 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.465456 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-data\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.465641 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.465486 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-podinfo\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.465641 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.465526 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-envoy\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.465641 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.465571 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-certs\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.566486 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.566445 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4llpd\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-kube-api-access-4llpd\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.566700 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.566508 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-token\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.566700 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.566545 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istiod-ca-cert\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.566700 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.566571 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-credential-socket\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.566700 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.566619 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-socket\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.566700 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.566676 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-data\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.567032 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.566704 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-podinfo\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.567032 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.566740 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-envoy\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.567032 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.566786 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-certs\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.567239 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.567216 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-credential-socket\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.567311 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.567261 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-socket\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.567311 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.567299 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-certs\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.567407 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.567319 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-data\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.567407 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.567381 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istiod-ca-cert\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.569362 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.569337 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-envoy\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.569582 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.569565 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-podinfo\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.579514 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.579485 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-token\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.581084 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.581065 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4llpd\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-kube-api-access-4llpd\") pod \"e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.684176 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.684079 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:12.830804 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.830774 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz"] Jul 09 14:32:12.831928 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:32:12.831903 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bf6039_a317_4cbd_8dfc_c7302d9459e6.slice/crio-aab1446fba19f5cc73d4cea522e7e6aff737a96c92ecbb06a8a64f937b759189 WatchSource:0}: Error finding container aab1446fba19f5cc73d4cea522e7e6aff737a96c92ecbb06a8a64f937b759189: Status 404 returned error can't find the container with id aab1446fba19f5cc73d4cea522e7e6aff737a96c92ecbb06a8a64f937b759189 Jul 09 14:32:12.834180 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.834143 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:12.834257 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.834227 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:12.834297 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:12.834284 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:13.480565 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:13.480521 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" event={"ID":"34bf6039-a317-4cbd-8dfc-c7302d9459e6","Type":"ContainerStarted","Data":"731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0"} Jul 09 14:32:13.480990 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:13.480628 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" event={"ID":"34bf6039-a317-4cbd-8dfc-c7302d9459e6","Type":"ContainerStarted","Data":"aab1446fba19f5cc73d4cea522e7e6aff737a96c92ecbb06a8a64f937b759189"} Jul 09 14:32:13.506235 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:13.506182 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" podStartSLOduration=1.5061677169999999 podStartE2EDuration="1.506167717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:13.505173837 +0000 UTC m=+2197.359125327" watchObservedRunningTime="2026-07-09 14:32:13.506167717 +0000 UTC m=+2197.360119192" Jul 09 14:32:13.685125 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:13.685081 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:13.690228 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:13.690200 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:14.488305 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:14.488269 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:14.489484 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:14.489462 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:15.641656 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:15.641603 2575 patch_prober.go:28] interesting pod/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:32:15.642315 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:15.642284 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:32:16.698441 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.698376 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:32:16.809677 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.809645 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istiod-ca-cert\") pod \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " Jul 09 14:32:16.809908 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.809705 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-token\") pod \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " Jul 09 14:32:16.809908 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.809756 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-certs\") pod \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " Jul 09 14:32:16.809908 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.809790 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-socket\") pod \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " Jul 09 14:32:16.809908 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.809818 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-podinfo\") pod \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " Jul 09 14:32:16.809908 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.809885 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-data\") pod \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " Jul 09 14:32:16.810202 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.809974 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-credential-socket\") pod \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " Jul 09 14:32:16.810202 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810052 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svr5f\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-kube-api-access-svr5f\") pod \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " Jul 09 14:32:16.810202 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810093 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-envoy\") pod \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\" (UID: \"d08b5ae8-8dce-4aec-ba2b-17e47d30e416\") " Jul 09 14:32:16.810202 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810116 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d08b5ae8-8dce-4aec-ba2b-17e47d30e416" (UID: "d08b5ae8-8dce-4aec-ba2b-17e47d30e416"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:32:16.810400 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810219 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d08b5ae8-8dce-4aec-ba2b-17e47d30e416" (UID: "d08b5ae8-8dce-4aec-ba2b-17e47d30e416"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:16.810400 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810302 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d08b5ae8-8dce-4aec-ba2b-17e47d30e416" (UID: "d08b5ae8-8dce-4aec-ba2b-17e47d30e416"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:16.810400 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810312 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d08b5ae8-8dce-4aec-ba2b-17e47d30e416" (UID: "d08b5ae8-8dce-4aec-ba2b-17e47d30e416"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:16.810564 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810473 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:16.810564 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810493 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:16.810564 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810509 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:16.810564 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810521 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:16.810789 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.810766 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-data" (OuterVolumeSpecName: "istio-data") pod "d08b5ae8-8dce-4aec-ba2b-17e47d30e416" (UID: "d08b5ae8-8dce-4aec-ba2b-17e47d30e416"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:16.812187 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.812162 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-token" (OuterVolumeSpecName: "istio-token") pod "d08b5ae8-8dce-4aec-ba2b-17e47d30e416" (UID: "d08b5ae8-8dce-4aec-ba2b-17e47d30e416"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:32:16.812334 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.812314 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d08b5ae8-8dce-4aec-ba2b-17e47d30e416" (UID: "d08b5ae8-8dce-4aec-ba2b-17e47d30e416"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:16.812544 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.812516 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-kube-api-access-svr5f" (OuterVolumeSpecName: "kube-api-access-svr5f") pod "d08b5ae8-8dce-4aec-ba2b-17e47d30e416" (UID: "d08b5ae8-8dce-4aec-ba2b-17e47d30e416"). InnerVolumeSpecName "kube-api-access-svr5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:32:16.812752 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.812730 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d08b5ae8-8dce-4aec-ba2b-17e47d30e416" (UID: "d08b5ae8-8dce-4aec-ba2b-17e47d30e416"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:32:16.911312 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.911272 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:16.911312 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.911300 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:16.911312 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.911309 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:16.911312 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.911317 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:16.911618 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:16.911329 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svr5f\" (UniqueName: \"kubernetes.io/projected/d08b5ae8-8dce-4aec-ba2b-17e47d30e416-kube-api-access-svr5f\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:17.508358 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.508320 2575 generic.go:356] "Generic (PLEG): container finished" podID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerID="ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526" exitCode=0 Jul 09 14:32:17.508531 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.508368 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" Jul 09 14:32:17.508531 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.508401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" event={"ID":"d08b5ae8-8dce-4aec-ba2b-17e47d30e416","Type":"ContainerDied","Data":"ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526"} Jul 09 14:32:17.508531 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.508442 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8" event={"ID":"d08b5ae8-8dce-4aec-ba2b-17e47d30e416","Type":"ContainerDied","Data":"5e16a2b5ac52e379fcbde71d208411dabea385a8f87adfc6e00d4601d730f52b"} Jul 09 14:32:17.508531 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.508463 2575 scope.go:117] "RemoveContainer" containerID="ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526" Jul 09 14:32:17.518931 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.518911 2575 scope.go:117] "RemoveContainer" containerID="deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa" Jul 09 14:32:17.528640 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.528619 2575 scope.go:117] "RemoveContainer" containerID="ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526" Jul 09 14:32:17.528944 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:32:17.528923 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526\": container with ID starting with ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526 not found: ID does not exist" containerID="ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526" Jul 09 14:32:17.529001 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.528956 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526"} err="failed to get container status \"ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526\": rpc error: code = NotFound desc = could not find container \"ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526\": container with ID starting with ba5eecbff40acafcb17b5a56a45150b3f84b8a561afc0cd2cbc03ead5fc12526 not found: ID does not exist" Jul 09 14:32:17.529001 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.528976 2575 scope.go:117] "RemoveContainer" containerID="deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa" Jul 09 14:32:17.529227 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:32:17.529206 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa\": container with ID starting with deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa not found: ID does not exist" containerID="deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa" Jul 09 14:32:17.529278 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.529234 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa"} err="failed to get container status \"deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa\": rpc error: code = NotFound desc = could not find container \"deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa\": container with ID starting with deb951e2a1269b5ec5478d69c97e2132b6bc12c0800998ad8cd09c96321334fa not found: ID does not exist" Jul 09 14:32:17.536065 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.536029 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8"] Jul 09 14:32:17.545820 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:17.545780 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f1a2e7-openshift-default-5cc698bb79-k9js8"] Jul 09 14:32:18.638722 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:18.638692 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" path="/var/lib/kubelet/pods/d08b5ae8-8dce-4aec-ba2b-17e47d30e416/volumes" Jul 09 14:32:20.690489 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.690449 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2"] Jul 09 14:32:20.690974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.690955 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" Jul 09 14:32:20.691037 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.690978 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" Jul 09 14:32:20.691037 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.690996 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" Jul 09 14:32:20.691037 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.691001 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" Jul 09 14:32:20.691135 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.691069 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" Jul 09 14:32:20.694642 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.694620 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.700389 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.700362 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-7c2ea1-openshift-default-dockercfg-swv9s\"" Jul 09 14:32:20.708970 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.708927 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2"] Jul 09 14:32:20.747508 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.747467 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-data\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.747709 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.747509 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-socket\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.747709 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.747586 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-envoy\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.747709 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.747671 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istiod-ca-cert\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.747881 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.747719 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-token\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.747881 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.747795 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-podinfo\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.747997 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.747907 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-credential-socket\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.747997 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.747949 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmlvz\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-kube-api-access-bmlvz\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.748167 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.748010 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-certs\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849235 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849197 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-token\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849412 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849274 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-podinfo\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849412 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849332 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-credential-socket\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849412 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849360 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlvz\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-kube-api-access-bmlvz\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849412 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849400 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-certs\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849611 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849443 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-data\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849611 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849467 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-socket\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849611 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-envoy\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849611 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849544 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istiod-ca-cert\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.849899 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849868 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-credential-socket\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.850039 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.849940 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-certs\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.850039 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.850014 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-socket\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.850172 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.850075 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-data\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.850364 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.850318 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istiod-ca-cert\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.852151 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.852128 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-envoy\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.852374 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.852356 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-podinfo\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.858368 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.858337 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-token\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:20.859167 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:20.859143 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmlvz\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-kube-api-access-bmlvz\") pod \"e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:21.005825 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:21.005723 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:21.160311 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:21.160282 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2"] Jul 09 14:32:21.162165 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:32:21.162135 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc263b1_b13c_47f5_8a25_3ec738fb4b23.slice/crio-384cc908d2a9455d242aeb301a7e31e5c6b6d2e05b8b290d0e70f635a4ec2b6e WatchSource:0}: Error finding container 384cc908d2a9455d242aeb301a7e31e5c6b6d2e05b8b290d0e70f635a4ec2b6e: Status 404 returned error can't find the container with id 384cc908d2a9455d242aeb301a7e31e5c6b6d2e05b8b290d0e70f635a4ec2b6e Jul 09 14:32:21.164534 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:21.164500 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:21.164613 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:21.164573 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:21.164613 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:21.164600 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:21.535828 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:21.535785 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" event={"ID":"7cc263b1-b13c-47f5-8a25-3ec738fb4b23","Type":"ContainerStarted","Data":"bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0"} Jul 09 14:32:21.536069 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:21.535844 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" event={"ID":"7cc263b1-b13c-47f5-8a25-3ec738fb4b23","Type":"ContainerStarted","Data":"384cc908d2a9455d242aeb301a7e31e5c6b6d2e05b8b290d0e70f635a4ec2b6e"} Jul 09 14:32:21.557762 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:21.557704 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" podStartSLOduration=1.557687273 podStartE2EDuration="1.557687273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:21.557542815 +0000 UTC m=+2205.411494301" watchObservedRunningTime="2026-07-09 14:32:21.557687273 +0000 UTC m=+2205.411638749" Jul 09 14:32:22.006999 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:22.006905 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:22.012024 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:22.011999 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:22.542212 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:22.542179 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:22.543155 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:22.543132 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:30.529034 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:30.528987 2575 patch_prober.go:28] interesting pod/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:32:30.529438 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:30.529078 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:32:31.599674 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:31.599590 2575 generic.go:356] "Generic (PLEG): container finished" podID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerID="731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0" exitCode=0 Jul 09 14:32:31.599674 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:31.599660 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" event={"ID":"34bf6039-a317-4cbd-8dfc-c7302d9459e6","Type":"ContainerDied","Data":"731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0"} Jul 09 14:32:31.600138 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:31.600092 2575 scope.go:117] "RemoveContainer" containerID="731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0" Jul 09 14:32:31.601103 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:31.601072 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:31.601175 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:31.601140 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:31.601212 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:31.601177 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:32.609905 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:32.609870 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" event={"ID":"34bf6039-a317-4cbd-8dfc-c7302d9459e6","Type":"ContainerStarted","Data":"efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba"} Jul 09 14:32:32.685651 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:32.685199 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:32.685651 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:32.685339 2575 patch_prober.go:28] interesting pod/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 14:32:32.685651 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:32.685383 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 14:32:33.684696 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:33.684656 2575 patch_prober.go:28] interesting pod/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 14:32:33.685318 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:33.685284 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 14:32:34.685550 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:34.685507 2575 patch_prober.go:28] interesting pod/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 14:32:34.686241 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:34.686205 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 14:32:35.684617 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:35.684573 2575 patch_prober.go:28] interesting pod/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 09 14:32:35.685050 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:35.685018 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 09 14:32:36.701538 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:36.700210 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:36.701538 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:36.701382 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:36.702533 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:36.702504 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:38.584077 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:38.584022 2575 patch_prober.go:28] interesting pod/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 14:32:38.584501 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:38.584119 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 14:32:40.666748 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:40.666641 2575 generic.go:356] "Generic (PLEG): container finished" podID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerID="bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0" exitCode=0 Jul 09 14:32:40.666748 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:40.666726 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" event={"ID":"7cc263b1-b13c-47f5-8a25-3ec738fb4b23","Type":"ContainerDied","Data":"bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0"} Jul 09 14:32:40.667378 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:40.667319 2575 scope.go:117] "RemoveContainer" containerID="bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0" Jul 09 14:32:40.668476 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:40.668444 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:40.668639 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:40.668536 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:40.668639 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:40.668585 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:41.675290 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:41.675259 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" event={"ID":"7cc263b1-b13c-47f5-8a25-3ec738fb4b23","Type":"ContainerStarted","Data":"2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79"} Jul 09 14:32:42.006910 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:42.006791 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:42.007091 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:42.007068 2575 patch_prober.go:28] interesting pod/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 09 14:32:42.007149 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:42.007131 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 09 14:32:43.006980 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:43.006941 2575 patch_prober.go:28] interesting pod/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 09 14:32:43.007468 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:43.007000 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 09 14:32:43.411980 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:43.411869 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz"] Jul 09 14:32:43.412195 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:43.412158 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" containerID="cri-o://efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba" gracePeriod=30 Jul 09 14:32:44.006640 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:44.006598 2575 patch_prober.go:28] interesting pod/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 09 14:32:44.006830 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:44.006656 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 09 14:32:45.006302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:45.006258 2575 patch_prober.go:28] interesting pod/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 09 14:32:45.006739 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:45.006324 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 09 14:32:46.010599 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:46.010557 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:46.011148 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:46.010949 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:46.011601 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:46.011582 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:48.677270 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.677199 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:48.738392 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.738351 2575 generic.go:356] "Generic (PLEG): container finished" podID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerID="efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba" exitCode=0 Jul 09 14:32:48.738600 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.738405 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" Jul 09 14:32:48.738600 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.738421 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" event={"ID":"34bf6039-a317-4cbd-8dfc-c7302d9459e6","Type":"ContainerDied","Data":"efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba"} Jul 09 14:32:48.738600 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.738475 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz" event={"ID":"34bf6039-a317-4cbd-8dfc-c7302d9459e6","Type":"ContainerDied","Data":"aab1446fba19f5cc73d4cea522e7e6aff737a96c92ecbb06a8a64f937b759189"} Jul 09 14:32:48.738600 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.738498 2575 scope.go:117] "RemoveContainer" containerID="efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba" Jul 09 14:32:48.748392 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.748370 2575 scope.go:117] "RemoveContainer" containerID="731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0" Jul 09 14:32:48.757301 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.757282 2575 scope.go:117] "RemoveContainer" containerID="efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba" Jul 09 14:32:48.757580 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:32:48.757562 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba\": container with ID starting with efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba not found: ID does not exist" containerID="efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba" Jul 09 14:32:48.757646 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.757588 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba"} err="failed to get container status \"efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba\": rpc error: code = NotFound desc = could not find container \"efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba\": container with ID starting with efd58dc8de575ccc0458bf9a590775f7af883a9701e3cbae077f7d6f2a6275ba not found: ID does not exist" Jul 09 14:32:48.757646 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.757606 2575 scope.go:117] "RemoveContainer" containerID="731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0" Jul 09 14:32:48.757949 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:32:48.757919 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0\": container with ID starting with 731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0 not found: ID does not exist" containerID="731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0" Jul 09 14:32:48.758008 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.757965 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0"} err="failed to get container status \"731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0\": rpc error: code = NotFound desc = could not find container \"731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0\": container with ID starting with 731bd1246026cb08c5325b4429f9b8855b40fd212f4ad83278f5b3b268af0fd0 not found: ID does not exist" Jul 09 14:32:48.764226 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764206 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-token\") pod \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " Jul 09 14:32:48.764331 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764235 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-envoy\") pod \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " Jul 09 14:32:48.764331 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764273 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4llpd\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-kube-api-access-4llpd\") pod \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " Jul 09 14:32:48.764331 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764318 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-certs\") pod \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " Jul 09 14:32:48.764479 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764353 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-credential-socket\") pod \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " Jul 09 14:32:48.764479 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764374 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-podinfo\") pod \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " Jul 09 14:32:48.764479 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764429 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-data\") pod \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " Jul 09 14:32:48.764479 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764460 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istiod-ca-cert\") pod \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " Jul 09 14:32:48.764682 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764521 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-socket\") pod \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\" (UID: \"34bf6039-a317-4cbd-8dfc-c7302d9459e6\") " Jul 09 14:32:48.764682 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764668 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "34bf6039-a317-4cbd-8dfc-c7302d9459e6" (UID: "34bf6039-a317-4cbd-8dfc-c7302d9459e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:48.764798 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764748 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "34bf6039-a317-4cbd-8dfc-c7302d9459e6" (UID: "34bf6039-a317-4cbd-8dfc-c7302d9459e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:48.764926 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.764902 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "34bf6039-a317-4cbd-8dfc-c7302d9459e6" (UID: "34bf6039-a317-4cbd-8dfc-c7302d9459e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:48.765075 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.765022 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:48.765075 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.765050 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "34bf6039-a317-4cbd-8dfc-c7302d9459e6" (UID: "34bf6039-a317-4cbd-8dfc-c7302d9459e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:32:48.765075 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.765052 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:48.765764 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.765734 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-data" (OuterVolumeSpecName: "istio-data") pod "34bf6039-a317-4cbd-8dfc-c7302d9459e6" (UID: "34bf6039-a317-4cbd-8dfc-c7302d9459e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:48.766797 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.766769 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-token" (OuterVolumeSpecName: "istio-token") pod "34bf6039-a317-4cbd-8dfc-c7302d9459e6" (UID: "34bf6039-a317-4cbd-8dfc-c7302d9459e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:32:48.766932 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.766772 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "34bf6039-a317-4cbd-8dfc-c7302d9459e6" (UID: "34bf6039-a317-4cbd-8dfc-c7302d9459e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:48.767401 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.767380 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-kube-api-access-4llpd" (OuterVolumeSpecName: "kube-api-access-4llpd") pod "34bf6039-a317-4cbd-8dfc-c7302d9459e6" (UID: "34bf6039-a317-4cbd-8dfc-c7302d9459e6"). InnerVolumeSpecName "kube-api-access-4llpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:32:48.767489 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.767468 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "34bf6039-a317-4cbd-8dfc-c7302d9459e6" (UID: "34bf6039-a317-4cbd-8dfc-c7302d9459e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:32:48.865837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.865793 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4llpd\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-kube-api-access-4llpd\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:48.865837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.865832 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:48.865837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.865842 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:48.866132 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.865882 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:48.866132 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.865891 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:48.866132 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.865900 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:48.866132 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:48.865908 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34bf6039-a317-4cbd-8dfc-c7302d9459e6-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:49.063707 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:49.063669 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz"] Jul 09 14:32:49.069404 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:49.069375 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-38f05c-openshift-default-6d64985bf-m6fbz"] Jul 09 14:32:50.493730 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:50.493683 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2"] Jul 09 14:32:50.494246 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:50.493979 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" containerID="cri-o://2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79" gracePeriod=30 Jul 09 14:32:50.638940 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:50.638907 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" path="/var/lib/kubelet/pods/34bf6039-a317-4cbd-8dfc-c7302d9459e6/volumes" Jul 09 14:32:51.356271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.356231 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8"] Jul 09 14:32:51.356757 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.356660 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" Jul 09 14:32:51.356757 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.356672 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" Jul 09 14:32:51.356757 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.356687 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" Jul 09 14:32:51.356757 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.356697 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" Jul 09 14:32:51.356941 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.356780 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" Jul 09 14:32:51.356941 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.356791 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08b5ae8-8dce-4aec-ba2b-17e47d30e416" containerName="istio-proxy" Jul 09 14:32:51.360154 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.360131 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.363112 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.363085 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-6b90ba-openshift-default-dockercfg-tjr4l\"" Jul 09 14:32:51.374990 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.374957 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8"] Jul 09 14:32:51.492641 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.492601 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-data\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.492837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.492642 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-token\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.492837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.492749 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k5gr\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-kube-api-access-2k5gr\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.492837 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.492784 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-envoy\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.493009 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.492839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-podinfo\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.493009 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.492919 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4225d141-ce24-4248-9cdc-78feca59c5c9-istiod-ca-cert\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.493009 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.492946 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-certs\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.493009 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.492986 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-socket\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.493161 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.493031 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-credential-socket\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.594062 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594025 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4225d141-ce24-4248-9cdc-78feca59c5c9-istiod-ca-cert\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.594551 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594084 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-certs\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.594551 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594139 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-socket\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.594551 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594199 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-credential-socket\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.594551 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594253 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-data\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.594551 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594277 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-token\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.594551 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594342 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k5gr\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-kube-api-access-2k5gr\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.594551 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594371 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-envoy\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.594551 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594418 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-podinfo\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.595046 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594568 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-certs\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.595046 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594657 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-data\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.595046 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594815 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-socket\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.595046 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.594962 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-credential-socket\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.595252 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.595189 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4225d141-ce24-4248-9cdc-78feca59c5c9-istiod-ca-cert\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.597397 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.597374 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-envoy\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.597705 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.597683 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-podinfo\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.605923 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.605895 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-token\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.606108 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.606086 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k5gr\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-kube-api-access-2k5gr\") pod \"e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.671276 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.671171 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:51.823760 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.823732 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8"] Jul 09 14:32:51.826346 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:32:51.826318 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4225d141_ce24_4248_9cdc_78feca59c5c9.slice/crio-b78f6b3b4cf30b1a550aed9c92019b59a1e8d7e4bf3f8fcbcbccea0d487a2a89 WatchSource:0}: Error finding container b78f6b3b4cf30b1a550aed9c92019b59a1e8d7e4bf3f8fcbcbccea0d487a2a89: Status 404 returned error can't find the container with id b78f6b3b4cf30b1a550aed9c92019b59a1e8d7e4bf3f8fcbcbccea0d487a2a89 Jul 09 14:32:51.829094 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.829053 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:51.829215 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.829142 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:51.829215 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:51.829187 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:32:52.765253 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:52.765218 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" event={"ID":"4225d141-ce24-4248-9cdc-78feca59c5c9","Type":"ContainerStarted","Data":"2c7150de0fb4bcde33706aaf10155bd9dff1da9e082e93d397967275b22b71ea"} Jul 09 14:32:52.765672 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:52.765279 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" event={"ID":"4225d141-ce24-4248-9cdc-78feca59c5c9","Type":"ContainerStarted","Data":"b78f6b3b4cf30b1a550aed9c92019b59a1e8d7e4bf3f8fcbcbccea0d487a2a89"} Jul 09 14:32:52.787950 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:52.787894 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" podStartSLOduration=1.78787794 podStartE2EDuration="1.78787794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:52.785929041 +0000 UTC m=+2236.639880517" watchObservedRunningTime="2026-07-09 14:32:52.78787794 +0000 UTC m=+2236.641829450" Jul 09 14:32:53.672209 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:53.672163 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:53.677505 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:53.677470 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:53.771003 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:53.770967 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:53.772212 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:53.772190 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:32:55.749299 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.749264 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:55.787330 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.787296 2575 generic.go:356] "Generic (PLEG): container finished" podID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerID="2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79" exitCode=0 Jul 09 14:32:55.787528 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.787426 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" Jul 09 14:32:55.787528 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.787457 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" event={"ID":"7cc263b1-b13c-47f5-8a25-3ec738fb4b23","Type":"ContainerDied","Data":"2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79"} Jul 09 14:32:55.787614 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.787551 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2" event={"ID":"7cc263b1-b13c-47f5-8a25-3ec738fb4b23","Type":"ContainerDied","Data":"384cc908d2a9455d242aeb301a7e31e5c6b6d2e05b8b290d0e70f635a4ec2b6e"} Jul 09 14:32:55.787614 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.787578 2575 scope.go:117] "RemoveContainer" containerID="2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79" Jul 09 14:32:55.797797 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.797748 2575 scope.go:117] "RemoveContainer" containerID="bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0" Jul 09 14:32:55.807505 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.807484 2575 scope.go:117] "RemoveContainer" containerID="2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79" Jul 09 14:32:55.807837 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:32:55.807813 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79\": container with ID starting with 2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79 not found: ID does not exist" containerID="2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79" Jul 09 14:32:55.807974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.807868 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79"} err="failed to get container status \"2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79\": rpc error: code = NotFound desc = could not find container \"2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79\": container with ID starting with 2b06a8a1eca61ce6d8f236b548985a43083ac5b52229fa8643b066d7ba6bec79 not found: ID does not exist" Jul 09 14:32:55.807974 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.807898 2575 scope.go:117] "RemoveContainer" containerID="bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0" Jul 09 14:32:55.808231 ip-10-0-128-190 kubenswrapper[2575]: E0709 14:32:55.808208 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0\": container with ID starting with bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0 not found: ID does not exist" containerID="bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0" Jul 09 14:32:55.808322 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.808237 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0"} err="failed to get container status \"bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0\": rpc error: code = NotFound desc = could not find container \"bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0\": container with ID starting with bda0267aefb7b896e3f49d7786359db5a5e23cb7ba019180fe49edfd6907b0c0 not found: ID does not exist" Jul 09 14:32:55.840862 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.840821 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-certs\") pod \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " Jul 09 14:32:55.841066 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.840903 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-token\") pod \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " Jul 09 14:32:55.841066 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.840931 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-envoy\") pod \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " Jul 09 14:32:55.841066 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.840954 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-credential-socket\") pod \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " Jul 09 14:32:55.841066 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.841031 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-socket\") pod \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " Jul 09 14:32:55.841066 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.841051 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-data\") pod \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " Jul 09 14:32:55.841484 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.841081 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istiod-ca-cert\") pod \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " Jul 09 14:32:55.841484 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.841127 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmlvz\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-kube-api-access-bmlvz\") pod \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " Jul 09 14:32:55.841484 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.841134 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7cc263b1-b13c-47f5-8a25-3ec738fb4b23" (UID: "7cc263b1-b13c-47f5-8a25-3ec738fb4b23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:55.841484 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.841158 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-podinfo\") pod \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\" (UID: \"7cc263b1-b13c-47f5-8a25-3ec738fb4b23\") " Jul 09 14:32:55.841484 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.841402 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7cc263b1-b13c-47f5-8a25-3ec738fb4b23" (UID: "7cc263b1-b13c-47f5-8a25-3ec738fb4b23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:55.841738 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.841588 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7cc263b1-b13c-47f5-8a25-3ec738fb4b23" (UID: "7cc263b1-b13c-47f5-8a25-3ec738fb4b23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:32:55.842137 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.841959 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-data" (OuterVolumeSpecName: "istio-data") pod "7cc263b1-b13c-47f5-8a25-3ec738fb4b23" (UID: "7cc263b1-b13c-47f5-8a25-3ec738fb4b23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:55.842137 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.842065 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:55.842137 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.842080 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:55.842137 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.842092 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:55.842137 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.842103 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:55.842564 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.842300 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7cc263b1-b13c-47f5-8a25-3ec738fb4b23" (UID: "7cc263b1-b13c-47f5-8a25-3ec738fb4b23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:55.844176 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.844150 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-token" (OuterVolumeSpecName: "istio-token") pod "7cc263b1-b13c-47f5-8a25-3ec738fb4b23" (UID: "7cc263b1-b13c-47f5-8a25-3ec738fb4b23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:32:55.844176 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.844154 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7cc263b1-b13c-47f5-8a25-3ec738fb4b23" (UID: "7cc263b1-b13c-47f5-8a25-3ec738fb4b23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:32:55.844176 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.844160 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7cc263b1-b13c-47f5-8a25-3ec738fb4b23" (UID: "7cc263b1-b13c-47f5-8a25-3ec738fb4b23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:32:55.844370 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.844351 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-kube-api-access-bmlvz" (OuterVolumeSpecName: "kube-api-access-bmlvz") pod "7cc263b1-b13c-47f5-8a25-3ec738fb4b23" (UID: "7cc263b1-b13c-47f5-8a25-3ec738fb4b23"). InnerVolumeSpecName "kube-api-access-bmlvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:32:55.943642 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.943585 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:55.943642 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.943635 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmlvz\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-kube-api-access-bmlvz\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:55.943642 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.943651 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:55.943985 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.943664 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:55.943985 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:55.943678 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc263b1-b13c-47f5-8a25-3ec738fb4b23-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:32:56.113791 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:56.113757 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2"] Jul 09 14:32:56.118513 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:56.118483 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-7c2ea1-openshift-default-75bc46f6c5-c4cj2"] Jul 09 14:32:56.638271 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:56.638191 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" path="/var/lib/kubelet/pods/7cc263b1-b13c-47f5-8a25-3ec738fb4b23/volumes" Jul 09 14:32:59.823565 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.823527 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr"] Jul 09 14:32:59.824286 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.824266 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" Jul 09 14:32:59.824338 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.824290 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" Jul 09 14:32:59.824441 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.824430 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" Jul 09 14:32:59.824478 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.824444 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" Jul 09 14:32:59.824478 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.824455 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="34bf6039-a317-4cbd-8dfc-c7302d9459e6" containerName="istio-proxy" Jul 09 14:32:59.828298 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.828275 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:32:59.831598 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.831573 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-11d0ca-openshift-default-dockercfg-88hwh\"" Jul 09 14:32:59.849534 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.849498 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr"] Jul 09 14:32:59.981818 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.981768 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66dc63a8-e866-414b-b41f-b47190a35797-istio-podinfo\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:32:59.981818 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.981819 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-istio-token\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:32:59.982067 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.981844 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-certs\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:32:59.982067 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.981887 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66dc63a8-e866-414b-b41f-b47190a35797-istiod-ca-cert\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:32:59.982067 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.981979 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-credential-socket\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:32:59.982159 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.982098 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-data\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:32:59.982159 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.982149 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-socket\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:32:59.982247 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.982175 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ncr5\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-kube-api-access-7ncr5\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:32:59.982302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:32:59.982266 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-envoy\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.083636 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.083535 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-data\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.083636 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.083597 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-socket\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.083636 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.083623 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ncr5\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-kube-api-access-7ncr5\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.083955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.083671 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-envoy\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.083955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.083720 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66dc63a8-e866-414b-b41f-b47190a35797-istio-podinfo\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.083955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.083747 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-istio-token\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.083955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.083772 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-certs\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.083955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.083798 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66dc63a8-e866-414b-b41f-b47190a35797-istiod-ca-cert\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.084232 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.084009 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-credential-socket\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.084232 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.084046 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-data\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.084232 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.084126 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-socket\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.084403 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.084350 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-credential-socket\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.084465 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.084437 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-certs\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.084596 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.084574 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66dc63a8-e866-414b-b41f-b47190a35797-istiod-ca-cert\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.086260 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.086241 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-envoy\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.086586 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.086561 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66dc63a8-e866-414b-b41f-b47190a35797-istio-podinfo\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.108327 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.108291 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ncr5\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-kube-api-access-7ncr5\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.108327 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.108306 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-istio-token\") pod \"e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.148603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.148563 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:00.307775 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.307743 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr"] Jul 09 14:33:00.310868 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:33:00.310814 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66dc63a8_e866_414b_b41f_b47190a35797.slice/crio-2068d20ea0cee8095b2c7aeabf08834f505a006cec3c474274392b418b047706 WatchSource:0}: Error finding container 2068d20ea0cee8095b2c7aeabf08834f505a006cec3c474274392b418b047706: Status 404 returned error can't find the container with id 2068d20ea0cee8095b2c7aeabf08834f505a006cec3c474274392b418b047706 Jul 09 14:33:00.313312 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.313269 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:33:00.313405 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.313366 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:33:00.313442 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.313407 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:33:00.821388 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.821349 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" event={"ID":"66dc63a8-e866-414b-b41f-b47190a35797","Type":"ContainerStarted","Data":"dd385d793da9cd36fc62fd9df8afdfe6425cfb8d4e79140da91c4940ade97d42"} Jul 09 14:33:00.821553 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.821418 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" event={"ID":"66dc63a8-e866-414b-b41f-b47190a35797","Type":"ContainerStarted","Data":"2068d20ea0cee8095b2c7aeabf08834f505a006cec3c474274392b418b047706"} Jul 09 14:33:00.843264 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:00.843215 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" podStartSLOduration=1.843197427 podStartE2EDuration="1.843197427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:33:00.841657543 +0000 UTC m=+2244.695609032" watchObservedRunningTime="2026-07-09 14:33:00.843197427 +0000 UTC m=+2244.697148903" Jul 09 14:33:01.149436 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:01.149345 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:01.154955 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:01.154929 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:01.827757 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:01.827730 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:01.828697 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:01.828675 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:08.772977 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:08.772935 2575 patch_prober.go:28] interesting pod/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 09 14:33:08.773490 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:08.773004 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 09 14:33:08.871933 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:08.871898 2575 generic.go:356] "Generic (PLEG): container finished" podID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerID="2c7150de0fb4bcde33706aaf10155bd9dff1da9e082e93d397967275b22b71ea" exitCode=0 Jul 09 14:33:08.872119 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:08.871944 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" event={"ID":"4225d141-ce24-4248-9cdc-78feca59c5c9","Type":"ContainerDied","Data":"2c7150de0fb4bcde33706aaf10155bd9dff1da9e082e93d397967275b22b71ea"} Jul 09 14:33:08.872396 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:08.872381 2575 scope.go:117] "RemoveContainer" containerID="2c7150de0fb4bcde33706aaf10155bd9dff1da9e082e93d397967275b22b71ea" Jul 09 14:33:08.873447 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:08.873417 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:33:08.873504 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:08.873485 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:33:08.873541 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:08.873528 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:33:09.896023 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:09.895979 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" event={"ID":"4225d141-ce24-4248-9cdc-78feca59c5c9","Type":"ContainerStarted","Data":"ad354d29bcbbdfca4bc089f204df955d27db5ec5dff8fb1e12f200bc0524216c"} Jul 09 14:33:10.671791 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:10.671749 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:33:10.672017 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:10.671986 2575 patch_prober.go:28] interesting pod/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 09 14:33:10.672074 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:10.672035 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 09 14:33:11.672252 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:11.672209 2575 patch_prober.go:28] interesting pod/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 09 14:33:11.672655 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:11.672271 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 09 14:33:12.671611 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:12.671567 2575 patch_prober.go:28] interesting pod/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 09 14:33:12.671839 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:12.671638 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 09 14:33:13.672524 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:13.672477 2575 patch_prober.go:28] interesting pod/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 09 14:33:13.672970 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:13.672535 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 09 14:33:14.675754 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:14.675724 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:33:14.676371 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:14.676336 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:33:14.681135 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:14.681054 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:33:20.974185 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:20.974150 2575 generic.go:356] "Generic (PLEG): container finished" podID="66dc63a8-e866-414b-b41f-b47190a35797" containerID="dd385d793da9cd36fc62fd9df8afdfe6425cfb8d4e79140da91c4940ade97d42" exitCode=0 Jul 09 14:33:20.974635 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:20.974228 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" event={"ID":"66dc63a8-e866-414b-b41f-b47190a35797","Type":"ContainerDied","Data":"dd385d793da9cd36fc62fd9df8afdfe6425cfb8d4e79140da91c4940ade97d42"} Jul 09 14:33:20.974680 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:20.974643 2575 scope.go:117] "RemoveContainer" containerID="dd385d793da9cd36fc62fd9df8afdfe6425cfb8d4e79140da91c4940ade97d42" Jul 09 14:33:20.975686 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:20.975654 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:33:20.975801 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:20.975739 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:33:20.975801 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:20.975788 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 14:33:21.986478 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:21.986431 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" event={"ID":"66dc63a8-e866-414b-b41f-b47190a35797","Type":"ContainerStarted","Data":"bb4ffae7056b0ba321e541a0cc756e084c4450d3b400e2cf5019960ddc865e6e"} Jul 09 14:33:22.149293 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:22.149246 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:22.149530 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:22.149506 2575 patch_prober.go:28] interesting pod/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 09 14:33:22.149578 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:22.149557 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 09 14:33:23.149828 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:23.149793 2575 patch_prober.go:28] interesting pod/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 09 14:33:23.150269 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:23.149880 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 09 14:33:24.149431 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:24.149387 2575 patch_prober.go:28] interesting pod/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 09 14:33:24.149621 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:24.149448 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 09 14:33:25.149447 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:25.149404 2575 patch_prober.go:28] interesting pod/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 09 14:33:25.149881 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:25.149464 2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 09 14:33:26.153702 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:26.153670 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:26.154113 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:26.153953 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:33:26.154908 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:33:26.154886 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:36:18.914832 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:18.914749 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8"] Jul 09 14:36:18.915370 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:18.915031 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" containerID="cri-o://ad354d29bcbbdfca4bc089f204df955d27db5ec5dff8fb1e12f200bc0524216c" gracePeriod=30 Jul 09 14:36:24.063378 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.063344 2575 generic.go:356] "Generic (PLEG): container finished" podID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerID="ad354d29bcbbdfca4bc089f204df955d27db5ec5dff8fb1e12f200bc0524216c" exitCode=0 Jul 09 14:36:24.063799 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.063423 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" event={"ID":"4225d141-ce24-4248-9cdc-78feca59c5c9","Type":"ContainerDied","Data":"ad354d29bcbbdfca4bc089f204df955d27db5ec5dff8fb1e12f200bc0524216c"} Jul 09 14:36:24.063799 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.063491 2575 scope.go:117] "RemoveContainer" containerID="2c7150de0fb4bcde33706aaf10155bd9dff1da9e082e93d397967275b22b71ea" Jul 09 14:36:24.175105 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.175076 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:36:24.270191 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270162 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-token\") pod \"4225d141-ce24-4248-9cdc-78feca59c5c9\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " Jul 09 14:36:24.270364 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270205 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-envoy\") pod \"4225d141-ce24-4248-9cdc-78feca59c5c9\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " Jul 09 14:36:24.270364 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270249 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-certs\") pod \"4225d141-ce24-4248-9cdc-78feca59c5c9\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " Jul 09 14:36:24.270364 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270277 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-data\") pod \"4225d141-ce24-4248-9cdc-78feca59c5c9\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " Jul 09 14:36:24.270364 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270302 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-podinfo\") pod \"4225d141-ce24-4248-9cdc-78feca59c5c9\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " Jul 09 14:36:24.270364 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270347 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2k5gr\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-kube-api-access-2k5gr\") pod \"4225d141-ce24-4248-9cdc-78feca59c5c9\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " Jul 09 14:36:24.270637 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270379 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-socket\") pod \"4225d141-ce24-4248-9cdc-78feca59c5c9\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " Jul 09 14:36:24.270637 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270441 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-credential-socket\") pod \"4225d141-ce24-4248-9cdc-78feca59c5c9\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " Jul 09 14:36:24.270637 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270517 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4225d141-ce24-4248-9cdc-78feca59c5c9-istiod-ca-cert\") pod \"4225d141-ce24-4248-9cdc-78feca59c5c9\" (UID: \"4225d141-ce24-4248-9cdc-78feca59c5c9\") " Jul 09 14:36:24.270797 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270615 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4225d141-ce24-4248-9cdc-78feca59c5c9" (UID: "4225d141-ce24-4248-9cdc-78feca59c5c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:24.270797 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270679 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4225d141-ce24-4248-9cdc-78feca59c5c9" (UID: "4225d141-ce24-4248-9cdc-78feca59c5c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:24.270932 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270811 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4225d141-ce24-4248-9cdc-78feca59c5c9" (UID: "4225d141-ce24-4248-9cdc-78feca59c5c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:24.270987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270930 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:24.270987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270949 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:24.270987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.270963 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:24.271250 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.271215 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4225d141-ce24-4248-9cdc-78feca59c5c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4225d141-ce24-4248-9cdc-78feca59c5c9" (UID: "4225d141-ce24-4248-9cdc-78feca59c5c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:36:24.271321 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.271291 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "4225d141-ce24-4248-9cdc-78feca59c5c9" (UID: "4225d141-ce24-4248-9cdc-78feca59c5c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:24.272579 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.272558 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "4225d141-ce24-4248-9cdc-78feca59c5c9" (UID: "4225d141-ce24-4248-9cdc-78feca59c5c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:36:24.273101 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.273077 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4225d141-ce24-4248-9cdc-78feca59c5c9" (UID: "4225d141-ce24-4248-9cdc-78feca59c5c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:24.273197 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.273082 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4225d141-ce24-4248-9cdc-78feca59c5c9" (UID: "4225d141-ce24-4248-9cdc-78feca59c5c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:36:24.273258 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.273243 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-kube-api-access-2k5gr" (OuterVolumeSpecName: "kube-api-access-2k5gr") pod "4225d141-ce24-4248-9cdc-78feca59c5c9" (UID: "4225d141-ce24-4248-9cdc-78feca59c5c9"). InnerVolumeSpecName "kube-api-access-2k5gr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:36:24.371482 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.371439 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k5gr\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-kube-api-access-2k5gr\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:24.371482 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.371471 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4225d141-ce24-4248-9cdc-78feca59c5c9-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:24.371482 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.371483 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:24.371482 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.371490 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:24.371755 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.371497 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:24.371755 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:24.371505 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4225d141-ce24-4248-9cdc-78feca59c5c9-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:25.071336 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:25.071302 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" Jul 09 14:36:25.071770 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:25.071302 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8" event={"ID":"4225d141-ce24-4248-9cdc-78feca59c5c9","Type":"ContainerDied","Data":"b78f6b3b4cf30b1a550aed9c92019b59a1e8d7e4bf3f8fcbcbccea0d487a2a89"} Jul 09 14:36:25.071770 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:25.071429 2575 scope.go:117] "RemoveContainer" containerID="ad354d29bcbbdfca4bc089f204df955d27db5ec5dff8fb1e12f200bc0524216c" Jul 09 14:36:25.094187 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:25.094149 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8"] Jul 09 14:36:25.099398 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:25.099369 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-6b90ba-openshift-default-848b4c5c67-d9ql8"] Jul 09 14:36:25.884821 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:25.884778 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr"] Jul 09 14:36:25.885097 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:25.885072 2575 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" containerID="cri-o://bb4ffae7056b0ba321e541a0cc756e084c4450d3b400e2cf5019960ddc865e6e" gracePeriod=30 Jul 09 14:36:26.156982 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:26.156894 2575 patch_prober.go:28] interesting pod/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 14:36:26.156982 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:26.156968 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 14:36:26.645878 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:26.645824 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" path="/var/lib/kubelet/pods/4225d141-ce24-4248-9cdc-78feca59c5c9/volumes" Jul 09 14:36:26.710705 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:26.710662 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-786779555b-cwfxk_3118e8fc-e111-4e21-b3c1-912e37a282d3/authorino/0.log" Jul 09 14:36:31.109253 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.109209 2575 generic.go:356] "Generic (PLEG): container finished" podID="66dc63a8-e866-414b-b41f-b47190a35797" containerID="bb4ffae7056b0ba321e541a0cc756e084c4450d3b400e2cf5019960ddc865e6e" exitCode=0 Jul 09 14:36:31.109668 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.109277 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" event={"ID":"66dc63a8-e866-414b-b41f-b47190a35797","Type":"ContainerDied","Data":"bb4ffae7056b0ba321e541a0cc756e084c4450d3b400e2cf5019960ddc865e6e"} Jul 09 14:36:31.109668 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.109347 2575 scope.go:117] "RemoveContainer" containerID="dd385d793da9cd36fc62fd9df8afdfe6425cfb8d4e79140da91c4940ade97d42" Jul 09 14:36:31.144211 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.144147 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:36:31.238487 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.238449 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-certs\") pod \"66dc63a8-e866-414b-b41f-b47190a35797\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " Jul 09 14:36:31.238674 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.238497 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-envoy\") pod \"66dc63a8-e866-414b-b41f-b47190a35797\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " Jul 09 14:36:31.238674 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.238535 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66dc63a8-e866-414b-b41f-b47190a35797-istio-podinfo\") pod \"66dc63a8-e866-414b-b41f-b47190a35797\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " Jul 09 14:36:31.238674 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.238553 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-istio-token\") pod \"66dc63a8-e866-414b-b41f-b47190a35797\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " Jul 09 14:36:31.238674 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.238586 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ncr5\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-kube-api-access-7ncr5\") pod \"66dc63a8-e866-414b-b41f-b47190a35797\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " Jul 09 14:36:31.238911 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.238771 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "66dc63a8-e866-414b-b41f-b47190a35797" (UID: "66dc63a8-e866-414b-b41f-b47190a35797"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:31.238911 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.238775 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-credential-socket\") pod \"66dc63a8-e866-414b-b41f-b47190a35797\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " Jul 09 14:36:31.238911 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.238892 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66dc63a8-e866-414b-b41f-b47190a35797-istiod-ca-cert\") pod \"66dc63a8-e866-414b-b41f-b47190a35797\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " Jul 09 14:36:31.239078 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239046 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-data\") pod \"66dc63a8-e866-414b-b41f-b47190a35797\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " Jul 09 14:36:31.239142 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239091 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-socket\") pod \"66dc63a8-e866-414b-b41f-b47190a35797\" (UID: \"66dc63a8-e866-414b-b41f-b47190a35797\") " Jul 09 14:36:31.239438 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239062 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "66dc63a8-e866-414b-b41f-b47190a35797" (UID: "66dc63a8-e866-414b-b41f-b47190a35797"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:31.239438 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239230 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66dc63a8-e866-414b-b41f-b47190a35797-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "66dc63a8-e866-414b-b41f-b47190a35797" (UID: "66dc63a8-e866-414b-b41f-b47190a35797"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 14:36:31.239438 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239298 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "66dc63a8-e866-414b-b41f-b47190a35797" (UID: "66dc63a8-e866-414b-b41f-b47190a35797"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:31.239645 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239461 2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:31.239645 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239474 2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-workload-certs\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:31.239645 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239482 2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-credential-socket\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:31.239645 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239490 2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66dc63a8-e866-414b-b41f-b47190a35797-istiod-ca-cert\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:31.240025 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.239997 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-data" (OuterVolumeSpecName: "istio-data") pod "66dc63a8-e866-414b-b41f-b47190a35797" (UID: "66dc63a8-e866-414b-b41f-b47190a35797"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:31.241167 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.241145 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/66dc63a8-e866-414b-b41f-b47190a35797-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "66dc63a8-e866-414b-b41f-b47190a35797" (UID: "66dc63a8-e866-414b-b41f-b47190a35797"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 14:36:31.241426 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.241410 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-kube-api-access-7ncr5" (OuterVolumeSpecName: "kube-api-access-7ncr5") pod "66dc63a8-e866-414b-b41f-b47190a35797" (UID: "66dc63a8-e866-414b-b41f-b47190a35797"). InnerVolumeSpecName "kube-api-access-7ncr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:36:31.241520 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.241504 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-istio-token" (OuterVolumeSpecName: "istio-token") pod "66dc63a8-e866-414b-b41f-b47190a35797" (UID: "66dc63a8-e866-414b-b41f-b47190a35797"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 14:36:31.241587 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.241519 2575 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "66dc63a8-e866-414b-b41f-b47190a35797" (UID: "66dc63a8-e866-414b-b41f-b47190a35797"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 14:36:31.340284 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.340247 2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-envoy\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:31.340284 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.340283 2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66dc63a8-e866-414b-b41f-b47190a35797-istio-podinfo\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:31.340284 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.340293 2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-istio-token\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:31.340537 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.340305 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ncr5\" (UniqueName: \"kubernetes.io/projected/66dc63a8-e866-414b-b41f-b47190a35797-kube-api-access-7ncr5\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:31.340537 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.340315 2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66dc63a8-e866-414b-b41f-b47190a35797-istio-data\") on node \"ip-10-0-128-190.ec2.internal\" DevicePath \"\"" Jul 09 14:36:31.439651 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.439568 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wpzxq_859a3a7e-f0b7-4395-9c82-ee497dd62aba/manager/0.log" Jul 09 14:36:31.553255 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:31.553211 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6864bb696c-h54jr_ce2dd036-027e-4656-94a6-439c4740cf8d/manager/0.log" Jul 09 14:36:32.013039 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.012998 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74b946fd-p6m9d_1b28d7a4-503a-4a59-85c1-c433b1776eb8/copy-manifests/0.log" Jul 09 14:36:32.020126 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.020102 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74b946fd-p6m9d_1b28d7a4-503a-4a59-85c1-c433b1776eb8/manager/0.log" Jul 09 14:36:32.116310 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.116272 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" event={"ID":"66dc63a8-e866-414b-b41f-b47190a35797","Type":"ContainerDied","Data":"2068d20ea0cee8095b2c7aeabf08834f505a006cec3c474274392b418b047706"} Jul 09 14:36:32.116310 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.116297 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr" Jul 09 14:36:32.116810 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.116328 2575 scope.go:117] "RemoveContainer" containerID="bb4ffae7056b0ba321e541a0cc756e084c4450d3b400e2cf5019960ddc865e6e" Jul 09 14:36:32.141222 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.141189 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr"] Jul 09 14:36:32.145774 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.145747 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-11d0ca-openshift-default-79f959bcb7-lpxdr"] Jul 09 14:36:32.638384 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.638346 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66dc63a8-e866-414b-b41f-b47190a35797" path="/var/lib/kubelet/pods/66dc63a8-e866-414b-b41f-b47190a35797/volumes" Jul 09 14:36:32.989987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.989918 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr_4152889b-5503-4d27-a0ec-33d4722e2652/pull/0.log" Jul 09 14:36:32.996494 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:32.996463 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr_4152889b-5503-4d27-a0ec-33d4722e2652/extract/0.log" Jul 09 14:36:33.002786 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.002762 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr_4152889b-5503-4d27-a0ec-33d4722e2652/util/0.log" Jul 09 14:36:33.119640 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.119611 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774_ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4/util/0.log" Jul 09 14:36:33.126269 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.126244 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774_ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4/pull/0.log" Jul 09 14:36:33.132892 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.132843 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774_ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4/extract/0.log" Jul 09 14:36:33.257347 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.257264 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx_aaeb587b-9833-486c-a551-a8a1ee38a561/util/0.log" Jul 09 14:36:33.264994 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.264968 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx_aaeb587b-9833-486c-a551-a8a1ee38a561/pull/0.log" Jul 09 14:36:33.272252 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.272227 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx_aaeb587b-9833-486c-a551-a8a1ee38a561/extract/0.log" Jul 09 14:36:33.393367 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.393334 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8_fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1/util/0.log" Jul 09 14:36:33.400660 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.400636 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8_fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1/pull/0.log" Jul 09 14:36:33.406991 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.406967 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8_fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1/extract/0.log" Jul 09 14:36:33.532147 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.532115 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-786779555b-cwfxk_3118e8fc-e111-4e21-b3c1-912e37a282d3/authorino/0.log" Jul 09 14:36:33.886701 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:33.886619 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kd7cv_ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2/kuadrant-console-plugin/0.log" Jul 09 14:36:34.383283 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:34.383251 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xl7qg_4b1344c0-fe58-4542-98a3-330a040be406/manager/0.log" Jul 09 14:36:34.730199 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:34.730120 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h866g_2c20d428-334e-4a19-905a-67144babe143/istio-proxy/0.log" Jul 09 14:36:35.207587 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:35.207539 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2njfs_08e20f32-3d40-470c-b35f-7b288fb5bb66/istio-proxy/1.log" Jul 09 14:36:35.326047 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:35.326015 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-dp7mm_073f64b8-36fc-4f7e-9999-238a9c5ebb87/payload-pre-processing/0.log" Jul 09 14:36:35.445319 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:35.445271 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5lpjj_5b882159-647a-4daa-a7c7-24af8297d013/payload-processing/0.log" Jul 09 14:36:35.559804 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:35.559773 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dcfb69f8f-b6xsv_2f13c2b9-d38e-4af1-b7bd-3caf13d73f57/router/0.log" Jul 09 14:36:36.011369 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:36.011287 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-t26l7_a37f0ded-f0b7-4851-acfb-cec15d4f1a99/main/0.log" Jul 09 14:36:36.231987 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:36.231957 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbqr59_4778066a-e22b-4ef8-b641-d965eb648d8c/main/0.log" Jul 09 14:36:43.206394 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:43.206356 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k4zsb_9137871a-fadb-43ef-acb4-c5917ed17a72/global-pull-secret-syncer/0.log" Jul 09 14:36:43.435605 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:43.435572 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7f7k4_a875e964-90d0-43e9-8da1-c427093ead9e/konnectivity-agent/0.log" Jul 09 14:36:43.523768 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:43.523733 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-190.ec2.internal_b3de1e4a975b796018b205e668ce5af6/haproxy/0.log" Jul 09 14:36:48.301213 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.301179 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr_4152889b-5503-4d27-a0ec-33d4722e2652/extract/0.log" Jul 09 14:36:48.341197 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.341159 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr_4152889b-5503-4d27-a0ec-33d4722e2652/util/0.log" Jul 09 14:36:48.380079 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.380051 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596rslr_4152889b-5503-4d27-a0ec-33d4722e2652/pull/0.log" Jul 09 14:36:48.421603 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.421575 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774_ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4/extract/0.log" Jul 09 14:36:48.445370 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.445338 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774_ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4/util/0.log" Jul 09 14:36:48.471123 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.471085 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e02z774_ab40ba36-2ab1-41dc-a16e-4fc0f1f150c4/pull/0.log" Jul 09 14:36:48.501065 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.501036 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx_aaeb587b-9833-486c-a551-a8a1ee38a561/extract/0.log" Jul 09 14:36:48.531614 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.531579 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx_aaeb587b-9833-486c-a551-a8a1ee38a561/util/0.log" Jul 09 14:36:48.556731 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.556654 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z5mnx_aaeb587b-9833-486c-a551-a8a1ee38a561/pull/0.log" Jul 09 14:36:48.597172 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.597124 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8_fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1/extract/0.log" Jul 09 14:36:48.625533 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.625506 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8_fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1/util/0.log" Jul 09 14:36:48.654325 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.654297 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d45k8_fe992216-6c4e-4f24-9ab1-0cb62f7c8ee1/pull/0.log" Jul 09 14:36:48.898585 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.898495 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-786779555b-cwfxk_3118e8fc-e111-4e21-b3c1-912e37a282d3/authorino/0.log" Jul 09 14:36:48.992224 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:48.992191 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kd7cv_ba1cdfc3-34e6-4dac-80e1-6aa58fe7d8b2/kuadrant-console-plugin/0.log" Jul 09 14:36:49.273584 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:49.273555 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xl7qg_4b1344c0-fe58-4542-98a3-330a040be406/manager/0.log" Jul 09 14:36:50.939294 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:50.939261 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d29f91ba-3556-4603-bcee-4ba7a5e687a7/alertmanager/0.log" Jul 09 14:36:50.995297 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:50.995268 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d29f91ba-3556-4603-bcee-4ba7a5e687a7/config-reloader/0.log" Jul 09 14:36:51.057173 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.057144 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d29f91ba-3556-4603-bcee-4ba7a5e687a7/kube-rbac-proxy-web/0.log" Jul 09 14:36:51.099802 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.099780 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d29f91ba-3556-4603-bcee-4ba7a5e687a7/kube-rbac-proxy/0.log" Jul 09 14:36:51.125056 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.125030 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d29f91ba-3556-4603-bcee-4ba7a5e687a7/kube-rbac-proxy-metric/0.log" Jul 09 14:36:51.151932 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.151911 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d29f91ba-3556-4603-bcee-4ba7a5e687a7/prom-label-proxy/0.log" Jul 09 14:36:51.177076 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.177055 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d29f91ba-3556-4603-bcee-4ba7a5e687a7/init-config-reloader/0.log" Jul 09 14:36:51.221540 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.221458 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-5f6fl_9520fb30-076b-49c3-90d6-5d6b8f87285d/cluster-monitoring-operator/0.log" Jul 09 14:36:51.267986 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.267952 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5z4jh_5fefb73d-befb-4eac-ba98-719b13989a8b/kube-state-metrics/0.log" Jul 09 14:36:51.292243 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.292217 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5z4jh_5fefb73d-befb-4eac-ba98-719b13989a8b/kube-rbac-proxy-main/0.log" Jul 09 14:36:51.319491 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.319460 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5z4jh_5fefb73d-befb-4eac-ba98-719b13989a8b/kube-rbac-proxy-self/0.log" Jul 09 14:36:51.418315 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.418290 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7bf4x_54448918-e070-4b17-8bc3-010541c874b9/node-exporter/0.log" Jul 09 14:36:51.448488 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.448444 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7bf4x_54448918-e070-4b17-8bc3-010541c874b9/kube-rbac-proxy/0.log" Jul 09 14:36:51.480659 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.480572 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7bf4x_54448918-e070-4b17-8bc3-010541c874b9/init-textfile/0.log" Jul 09 14:36:51.690222 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.690192 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-jksvq_3be9c8d3-a431-4c72-94ab-dd15bf815990/kube-rbac-proxy-main/0.log" Jul 09 14:36:51.714104 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.714066 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-jksvq_3be9c8d3-a431-4c72-94ab-dd15bf815990/kube-rbac-proxy-self/0.log" Jul 09 14:36:51.741910 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.741806 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-jksvq_3be9c8d3-a431-4c72-94ab-dd15bf815990/openshift-state-metrics/0.log" Jul 09 14:36:51.986952 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:51.986923 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-rfvhz_18fa7a34-4f44-4ee7-81f5-9689e72ca51b/prometheus-operator/0.log" Jul 09 14:36:52.015689 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:52.015617 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-rfvhz_18fa7a34-4f44-4ee7-81f5-9689e72ca51b/kube-rbac-proxy/0.log" Jul 09 14:36:52.074435 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:52.074407 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d48595994-rkg6b_7cfeed82-6ea1-4547-b5d6-ed81cee08b22/telemeter-client/0.log" Jul 09 14:36:52.098697 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:52.098666 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d48595994-rkg6b_7cfeed82-6ea1-4547-b5d6-ed81cee08b22/reload/0.log" Jul 09 14:36:52.123100 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:52.123074 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d48595994-rkg6b_7cfeed82-6ea1-4547-b5d6-ed81cee08b22/kube-rbac-proxy/0.log" Jul 09 14:36:53.041579 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.041502 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq"] Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042102 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042120 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042134 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042143 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042151 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042159 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc263b1-b13c-47f5-8a25-3ec738fb4b23" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042205 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042213 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042223 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042231 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" Jul 09 14:36:53.042302 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042306 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4225d141-ce24-4248-9cdc-78feca59c5c9" containerName="istio-proxy" Jul 09 14:36:53.042658 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.042315 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="66dc63a8-e866-414b-b41f-b47190a35797" containerName="istio-proxy" Jul 09 14:36:53.044489 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.044473 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.047294 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.047271 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fq7xw\"/\"openshift-service-ca.crt\"" Jul 09 14:36:53.047428 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.047391 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fq7xw\"/\"kube-root-ca.crt\"" Jul 09 14:36:53.048331 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.048317 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fq7xw\"/\"default-dockercfg-vd84f\"" Jul 09 14:36:53.056132 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.056108 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq"] Jul 09 14:36:53.141711 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.141677 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt8jz\" (UniqueName: \"kubernetes.io/projected/84d23095-3431-4573-bb5b-9cec94127a70-kube-api-access-bt8jz\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.141917 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.141734 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-lib-modules\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.141917 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.141881 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-podres\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.142005 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.141916 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-proc\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.142005 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.141984 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-sys\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.242889 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.242835 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-sys\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.243080 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.242906 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt8jz\" (UniqueName: \"kubernetes.io/projected/84d23095-3431-4573-bb5b-9cec94127a70-kube-api-access-bt8jz\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.243080 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.242948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-lib-modules\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.243080 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.242975 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-sys\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.243080 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.243032 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-podres\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.243080 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.243063 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-proc\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.243330 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.243105 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-lib-modules\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.243330 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.243140 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-proc\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.243330 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.243167 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/84d23095-3431-4573-bb5b-9cec94127a70-podres\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.254612 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.254578 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt8jz\" (UniqueName: \"kubernetes.io/projected/84d23095-3431-4573-bb5b-9cec94127a70-kube-api-access-bt8jz\") pod \"perf-node-gather-daemonset-b2mkq\" (UID: \"84d23095-3431-4573-bb5b-9cec94127a70\") " pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.355149 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.355050 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:53.432935 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.432901 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-cngfc_e836fc41-ecd7-4b4d-83c1-55f05ec601bd/networking-console-plugin/0.log" Jul 09 14:36:53.503736 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:53.503705 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq"] Jul 09 14:36:53.505815 ip-10-0-128-190 kubenswrapper[2575]: W0709 14:36:53.505788 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod84d23095_3431_4573_bb5b_9cec94127a70.slice/crio-1b95183d468c9c6a1c81844bfc93136ec355bb000f1a95a0f25cd69de106ab76 WatchSource:0}: Error finding container 1b95183d468c9c6a1c81844bfc93136ec355bb000f1a95a0f25cd69de106ab76: Status 404 returned error can't find the container with id 1b95183d468c9c6a1c81844bfc93136ec355bb000f1a95a0f25cd69de106ab76 Jul 09 14:36:54.245724 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:54.245684 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" event={"ID":"84d23095-3431-4573-bb5b-9cec94127a70","Type":"ContainerStarted","Data":"58f9063328f8bb484ff38721655088ad6d8693400cc02b5b14ba4215fa8ba47b"} Jul 09 14:36:54.246151 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:54.245785 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:36:54.246151 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:54.245810 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" event={"ID":"84d23095-3431-4573-bb5b-9cec94127a70","Type":"ContainerStarted","Data":"1b95183d468c9c6a1c81844bfc93136ec355bb000f1a95a0f25cd69de106ab76"} Jul 09 14:36:54.263867 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:54.263806 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" podStartSLOduration=1.263790557 podStartE2EDuration="1.263790557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:36:54.260760789 +0000 UTC m=+2478.114712267" watchObservedRunningTime="2026-07-09 14:36:54.263790557 +0000 UTC m=+2478.117742033" Jul 09 14:36:55.170981 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:55.170927 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-f25wz_9a3f5b05-7f0a-4075-a4da-78796a325d8a/volume-data-source-validator/0.log" Jul 09 14:36:56.228880 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:56.228836 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gnkr9_92ccdc79-fdc6-48fe-a976-23b6362217c1/dns/0.log" Jul 09 14:36:56.256816 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:56.256782 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gnkr9_92ccdc79-fdc6-48fe-a976-23b6362217c1/kube-rbac-proxy/0.log" Jul 09 14:36:56.458118 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:56.458092 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xd872_45e4c2e5-2a9c-4508-a9d1-e2c274713331/dns-node-resolver/0.log" Jul 09 14:36:57.042164 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:57.042137 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5tgvd_232c0b60-887d-41bd-8563-67bd303f57e4/node-ca/0.log" Jul 09 14:36:58.066804 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:58.066771 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h866g_2c20d428-334e-4a19-905a-67144babe143/istio-proxy/0.log" Jul 09 14:36:58.327325 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:58.327239 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2njfs_08e20f32-3d40-470c-b35f-7b288fb5bb66/istio-proxy/0.log" Jul 09 14:36:58.356126 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:58.356097 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2njfs_08e20f32-3d40-470c-b35f-7b288fb5bb66/istio-proxy/1.log" Jul 09 14:36:58.394844 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:58.394813 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-dp7mm_073f64b8-36fc-4f7e-9999-238a9c5ebb87/payload-pre-processing/0.log" Jul 09 14:36:58.444724 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:58.444686 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5lpjj_5b882159-647a-4daa-a7c7-24af8297d013/payload-processing/0.log" Jul 09 14:36:58.469252 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:58.469220 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dcfb69f8f-b6xsv_2f13c2b9-d38e-4af1-b7bd-3caf13d73f57/router/0.log" Jul 09 14:36:59.203822 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:59.203796 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5xqw6_48bc75ef-a454-4f16-8861-a3ca613afd68/serve-healthcheck-canary/0.log" Jul 09 14:36:59.749219 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:59.749191 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9hskn_f6559b7d-0f35-49e5-a64b-b43d2c566bfe/kube-rbac-proxy/0.log" Jul 09 14:36:59.774142 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:59.774119 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9hskn_f6559b7d-0f35-49e5-a64b-b43d2c566bfe/exporter/0.log" Jul 09 14:36:59.798600 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:36:59.798575 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9hskn_f6559b7d-0f35-49e5-a64b-b43d2c566bfe/extractor/0.log" Jul 09 14:37:00.262549 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:00.262520 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fq7xw/perf-node-gather-daemonset-b2mkq" Jul 09 14:37:02.172487 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:02.172450 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wpzxq_859a3a7e-f0b7-4395-9c82-ee497dd62aba/manager/0.log" Jul 09 14:37:02.261427 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:02.261393 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6864bb696c-h54jr_ce2dd036-027e-4656-94a6-439c4740cf8d/manager/0.log" Jul 09 14:37:02.474889 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:02.474793 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74b946fd-p6m9d_1b28d7a4-503a-4a59-85c1-c433b1776eb8/manager/0.log" Jul 09 14:37:02.501547 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:02.501516 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74b946fd-p6m9d_1b28d7a4-503a-4a59-85c1-c433b1776eb8/copy-manifests/0.log" Jul 09 14:37:03.772115 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:03.772083 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h866g_2c20d428-334e-4a19-905a-67144babe143/istio-proxy/0.log" Jul 09 14:37:03.994282 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:03.994238 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2njfs_08e20f32-3d40-470c-b35f-7b288fb5bb66/istio-proxy/0.log" Jul 09 14:37:04.021726 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:04.021692 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2njfs_08e20f32-3d40-470c-b35f-7b288fb5bb66/istio-proxy/1.log" Jul 09 14:37:04.035123 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:04.035089 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c98bd85cf-5677g_53cfee3e-a549-4293-86e1-5dfb65a76cf6/manager/0.log" Jul 09 14:37:04.053616 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:04.053566 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-dp7mm_073f64b8-36fc-4f7e-9999-238a9c5ebb87/payload-pre-processing/0.log" Jul 09 14:37:04.090575 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:04.090546 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5lpjj_5b882159-647a-4daa-a7c7-24af8297d013/payload-processing/0.log" Jul 09 14:37:04.101208 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:04.101180 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ljf8b_212df6b7-3466-4c4c-84ef-071e017395cd/openshift-lws-operator/0.log" Jul 09 14:37:04.109204 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:04.109183 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dcfb69f8f-b6xsv_2f13c2b9-d38e-4af1-b7bd-3caf13d73f57/router/0.log" Jul 09 14:37:05.334046 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:05.334017 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h866g_2c20d428-334e-4a19-905a-67144babe143/istio-proxy/0.log" Jul 09 14:37:05.562158 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:05.562120 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2njfs_08e20f32-3d40-470c-b35f-7b288fb5bb66/istio-proxy/0.log" Jul 09 14:37:05.593658 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:05.593600 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-2njfs_08e20f32-3d40-470c-b35f-7b288fb5bb66/istio-proxy/1.log" Jul 09 14:37:05.634977 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:05.634946 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-dp7mm_073f64b8-36fc-4f7e-9999-238a9c5ebb87/payload-pre-processing/0.log" Jul 09 14:37:05.672598 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:05.672559 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5lpjj_5b882159-647a-4daa-a7c7-24af8297d013/payload-processing/0.log" Jul 09 14:37:05.687230 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:05.687204 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6dcfb69f8f-b6xsv_2f13c2b9-d38e-4af1-b7bd-3caf13d73f57/router/0.log" Jul 09 14:37:11.234956 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:11.234920 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs2sz_412a8a2b-3312-4a3f-be35-dbba3ef982eb/kube-multus-additional-cni-plugins/0.log" Jul 09 14:37:11.260058 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:11.260022 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs2sz_412a8a2b-3312-4a3f-be35-dbba3ef982eb/egress-router-binary-copy/0.log" Jul 09 14:37:11.286808 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:11.286761 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs2sz_412a8a2b-3312-4a3f-be35-dbba3ef982eb/cni-plugins/0.log" Jul 09 14:37:11.313167 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:11.313139 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs2sz_412a8a2b-3312-4a3f-be35-dbba3ef982eb/bond-cni-plugin/0.log" Jul 09 14:37:11.342218 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:11.342191 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs2sz_412a8a2b-3312-4a3f-be35-dbba3ef982eb/routeoverride-cni/0.log" Jul 09 14:37:11.367956 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:11.367925 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs2sz_412a8a2b-3312-4a3f-be35-dbba3ef982eb/whereabouts-cni-bincopy/0.log" Jul 09 14:37:11.394446 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:11.394419 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zs2sz_412a8a2b-3312-4a3f-be35-dbba3ef982eb/whereabouts-cni/0.log" Jul 09 14:37:11.435684 ip-10-0-128-190 kubenswrapper[2575]: I0709 14:37:11.435652 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-m2hqg_36f61295-2489-44e4-b59d-51d77bd5f40b/kube-multus/0.log"