Jul 15 19:13:39.479452 ip-10-0-140-236 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 19:13:39.479466 ip-10-0-140-236 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 19:13:39.479474 ip-10-0-140-236 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 19:13:39.480024 ip-10-0-140-236 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 19:13:49.500767 ip-10-0-140-236 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 19:13:49.500783 ip-10-0-140-236 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e8413065721f4d669c8311c34e7263f3 -- Jul 15 19:15:54.044901 ip-10-0-140-236 systemd[1]: Starting Kubernetes Kubelet... Jul 15 19:15:54.457050 ip-10-0-140-236 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:15:54.457050 ip-10-0-140-236 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 19:15:54.457050 ip-10-0-140-236 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:15:54.457050 ip-10-0-140-236 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 19:15:54.457050 ip-10-0-140-236 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:15:54.457839 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.457752 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 19:15:54.463369 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463340 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:15:54.463369 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463363 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:15:54.463369 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463369 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:15:54.463369 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463372 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:15:54.463369 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463376 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:15:54.463369 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463379 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463383 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463387 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463391 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463394 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463398 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463402 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463407 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463410 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463413 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463416 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463420 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463423 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463428 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463431 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463435 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463438 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463442 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463450 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463454 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:15:54.463731 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463458 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463462 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463466 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463469 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463473 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463477 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463480 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463484 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463488 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463492 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463497 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463503 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463508 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463512 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463517 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463521 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463525 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463529 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463533 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463537 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:15:54.464522 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463541 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463546 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463550 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463554 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463558 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463563 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463567 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463595 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463600 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463605 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463610 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463614 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463621 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463627 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463632 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463637 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463641 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463645 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463649 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:15:54.465383 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463653 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463658 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463664 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463668 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463672 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463676 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463680 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463685 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463689 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463693 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463698 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463702 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463706 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463710 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463714 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463718 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463722 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463726 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463732 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463739 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:15:54.466206 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463744 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463748 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463754 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463759 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463763 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463767 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463774 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463778 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463782 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463786 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463790 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463795 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463799 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463803 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.463806 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.464997 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465011 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465018 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465023 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:15:54.467053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465029 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465033 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465039 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465044 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465048 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465053 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465057 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465062 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465067 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465071 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465076 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465080 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465085 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465090 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465094 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465098 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465102 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465106 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465110 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465114 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:15:54.467894 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465118 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465122 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465126 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465130 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465133 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465137 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465141 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465145 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465149 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465153 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465157 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465161 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465166 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465170 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465174 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465178 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465182 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465186 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465192 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465196 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:15:54.468501 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465200 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465204 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465209 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465213 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465217 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465222 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465227 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465231 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465235 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465239 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465243 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465249 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465253 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465258 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465262 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465265 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465270 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465274 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465278 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465282 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:15:54.468991 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465286 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465290 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465294 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465298 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465302 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465306 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465311 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465316 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465321 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465326 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465330 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465334 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465338 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465343 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465347 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465355 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465361 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465365 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465369 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:15:54.469604 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465374 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465378 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465382 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465387 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465390 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465394 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465398 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465403 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465406 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465410 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465414 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465418 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465423 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465427 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465431 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.465435 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466105 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466115 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466119 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466124 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:15:54.470291 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466129 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466133 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466137 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466142 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466146 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466151 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466155 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466161 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466167 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466172 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466177 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466181 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466187 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466191 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466195 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466200 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466203 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466207 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466212 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466215 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:15:54.470798 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466220 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466223 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466227 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466231 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466236 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466240 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466244 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466248 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466253 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466257 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466261 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466266 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466270 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466274 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466278 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466282 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466286 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466291 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466295 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:15:54.471280 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466300 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466304 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466308 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466312 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466315 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466319 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466324 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466328 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466332 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466336 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466340 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466344 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466348 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466354 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466358 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466362 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466366 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466370 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466374 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466377 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:15:54.471828 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466381 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466388 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466393 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466398 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466402 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466407 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466413 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466418 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466423 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466427 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466430 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466434 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466438 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466442 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466446 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466450 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466454 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466458 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466462 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:15:54.472319 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466466 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466470 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466474 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466478 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466482 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466486 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466490 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466494 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466498 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466502 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466508 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466512 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466516 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466520 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466524 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466532 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.466536 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466659 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466672 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466681 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466687 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 19:15:54.472785 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466694 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466699 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466706 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466713 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466717 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466722 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466728 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466732 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466738 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466742 2562 flags.go:64] FLAG: --cgroup-root="" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466747 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466751 2562 flags.go:64] FLAG: --client-ca-file="" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466757 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466762 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466770 2562 flags.go:64] FLAG: --cluster-domain="" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466778 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466783 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466788 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466793 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466799 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466803 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466808 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466813 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466818 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 15 19:15:54.473272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466823 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466827 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466834 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466838 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466845 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466850 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466854 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466858 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466863 2562 flags.go:64] FLAG: --enable-server="true" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466868 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466874 2562 flags.go:64] FLAG: --event-burst="100" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466879 2562 flags.go:64] FLAG: --event-qps="50" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466883 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466888 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466892 2562 flags.go:64] FLAG: --eviction-hard="" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466898 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466903 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466907 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466912 2562 flags.go:64] FLAG: --eviction-soft="" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466917 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466921 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466927 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466931 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466941 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466946 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 19:15:54.473847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466950 2562 flags.go:64] FLAG: --feature-gates="" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466957 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466961 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466966 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466971 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466975 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466980 2562 flags.go:64] FLAG: --help="false" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466985 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466990 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.466996 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467001 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467006 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467011 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467016 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467020 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467025 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467030 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467034 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467039 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467043 2562 flags.go:64] FLAG: --kube-reserved="" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467048 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467052 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467056 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467060 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 19:15:54.474485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467064 2562 flags.go:64] FLAG: --lock-file="" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467068 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467073 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467078 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467086 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467091 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467098 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467102 2562 flags.go:64] FLAG: --logging-format="text" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467107 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467113 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467117 2562 flags.go:64] FLAG: --manifest-url="" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467121 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467129 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467134 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467140 2562 flags.go:64] FLAG: --max-pods="110" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467144 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467149 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467154 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467159 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467164 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467168 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467173 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467185 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467190 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467194 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 19:15:54.475098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467199 2562 flags.go:64] FLAG: --pod-cidr="" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467203 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467213 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467218 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467222 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467227 2562 flags.go:64] FLAG: --port="10250" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467231 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467236 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03c463f3bd2b84742" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467241 2562 flags.go:64] FLAG: --qos-reserved="" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467245 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467250 2562 flags.go:64] FLAG: --register-node="true" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467255 2562 flags.go:64] FLAG: --register-with-taints="" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467261 2562 flags.go:64] FLAG: --registry-burst="10" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467270 2562 flags.go:64] FLAG: --registry-qps="5" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467274 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467279 2562 flags.go:64] FLAG: --reserved-memory="" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467285 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467289 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467294 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467299 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467303 2562 flags.go:64] FLAG: --runonce="false" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467307 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467312 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467316 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467321 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 19:15:54.475706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467327 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467332 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467337 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467341 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467346 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467350 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467354 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467359 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467363 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467368 2562 flags.go:64] FLAG: --system-cgroups="" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467372 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467381 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467385 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467389 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467396 2562 flags.go:64] FLAG: --tls-min-version="" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467401 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467405 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467409 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467414 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467418 2562 flags.go:64] FLAG: --v="2" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467425 2562 flags.go:64] FLAG: --version="false" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467432 2562 flags.go:64] FLAG: --vmodule="" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467438 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.467443 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467642 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:15:54.476314 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467650 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467654 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467659 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467664 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467669 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467674 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467677 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467684 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467688 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467692 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467696 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467700 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467704 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467708 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467712 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467716 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467720 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467726 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467732 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467736 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:15:54.476926 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467741 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467746 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467750 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467754 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467757 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467762 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467766 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467771 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467775 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467781 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467785 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467789 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467793 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467797 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467800 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467805 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467809 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467813 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467817 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467823 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:15:54.477405 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467852 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467858 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467863 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467868 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467872 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467876 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467881 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467885 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467889 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467893 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467897 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467900 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467904 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467908 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467912 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467916 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467920 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467924 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467929 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467934 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:15:54.477905 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467938 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467943 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467948 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467955 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467961 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467965 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467969 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467974 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467978 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467983 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467987 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467993 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.467996 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468001 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468005 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468009 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468013 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468017 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468028 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:15:54.478387 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468032 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468039 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468044 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468048 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468052 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468057 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468061 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468064 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468069 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468073 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468077 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468081 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468087 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468093 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468097 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468101 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468106 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468110 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:15:54.478896 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.468114 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.469006 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.476045 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.476063 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.476077 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.476091 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476150 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476155 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476159 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476161 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476164 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476167 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476171 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476174 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:15:54.479343 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476176 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476179 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476181 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476184 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476186 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476189 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476191 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476194 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476197 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476199 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476202 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476204 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476207 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476209 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476211 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476215 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476217 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476219 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476222 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476224 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:15:54.479709 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476227 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476235 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476238 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476241 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476243 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476245 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476248 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476250 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476252 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476255 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476257 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476261 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476264 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476268 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476271 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476274 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476277 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476291 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:15:54.480194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476295 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476298 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476300 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476303 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476306 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476308 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476311 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476313 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476316 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476318 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476321 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476323 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476326 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476328 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476331 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476335 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476344 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476348 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476351 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476353 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:15:54.480669 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476356 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476358 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476361 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476363 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476365 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476368 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476370 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476372 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476375 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476377 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476379 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476382 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476384 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476386 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476389 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476391 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476393 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476396 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476398 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476400 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:15:54.481136 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476403 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476405 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476407 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476409 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476411 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476414 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476416 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476420 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476422 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476430 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476432 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476435 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476437 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.476442 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476592 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:15:54.481625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476598 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476601 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476603 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476606 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476608 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476610 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476613 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476615 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476618 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476620 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476623 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476625 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476628 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476630 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476632 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476635 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476637 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476639 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476641 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476644 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:15:54.481985 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476646 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476649 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476651 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476653 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476656 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476658 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476662 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476672 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476675 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476678 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476680 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476683 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476685 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476688 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476690 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476692 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476694 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476697 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476699 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:15:54.482449 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476701 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476705 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476718 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476721 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476724 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476727 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476730 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476732 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476735 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476737 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476740 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476742 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476745 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476747 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476749 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476752 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476754 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476756 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476759 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476762 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:15:54.482965 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476764 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476772 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476775 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476777 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476779 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476782 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476784 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476787 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476789 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476791 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476794 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476796 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476798 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476801 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476803 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476806 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476808 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476811 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476813 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476815 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:15:54.483433 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476818 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476820 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476823 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476825 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476828 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476830 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476833 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476835 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476837 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476840 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476844 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476847 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476850 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476852 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476862 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476864 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476867 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476869 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:15:54.483958 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:54.476871 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:15:54.484386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.476875 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:15:54.484386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.477012 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 19:15:54.484386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.479077 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 19:15:54.484386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.480640 2562 server.go:1019] "Starting client certificate rotation" Jul 15 19:15:54.484386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.480748 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:15:54.484386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.480814 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:15:54.484814 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.484793 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:15:54.488860 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.488825 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:15:54.497389 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.497372 2562 log.go:25] "Validated CRI v1 runtime API" Jul 15 19:15:54.499354 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.499339 2562 log.go:25] "Validated CRI v1 image API" Jul 15 19:15:54.500332 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.500317 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 19:15:54.505755 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.505725 2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 86c8ade4-1c9e-4d73-b3d3-b86a719ced45:/dev/nvme0n1p4 f9e1e477-808e-4df0-850f-30b07308e563:/dev/nvme0n1p3] Jul 15 19:15:54.505829 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.505753 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 19:15:54.510778 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.510660 2562 manager.go:216] Machine: {Timestamp:2026-07-15 19:15:54.509355797 +0000 UTC m=+0.346060565 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3203776 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26c6d03c99f64685c575b904fb5ab0 SystemUUID:ec26c6d0-3c99-f646-85c5-75b904fb5ab0 BootID:e8413065-721f-4d66-9c83-11c34e7263f3 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:52:93:ee:d6:21 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:52:93:ee:d6:21 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:d4:74:96:28:d3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 19:15:54.510778 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.510766 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 19:15:54.510932 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.510875 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 19:15:54.514001 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.513974 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 19:15:54.514144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.514003 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 19:15:54.514781 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.514772 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 19:15:54.514819 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.514784 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 19:15:54.514819 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.514791 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:15:54.515475 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.515466 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:15:54.515898 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.515879 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:15:54.516765 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.516752 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 19:15:54.519557 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.519542 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:15:54.519719 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.519707 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 19:15:54.519785 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.519754 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 15 19:15:54.519785 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.519770 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 19:15:54.519876 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.519790 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 19:15:54.519876 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.519803 2562 kubelet.go:398] "Adding apiserver pod source" Jul 15 19:15:54.519876 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.519815 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 19:15:54.520254 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.520239 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:15:54.520316 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.520262 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:15:54.523039 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.523020 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 19:15:54.524833 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.524817 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 19:15:54.524906 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.524856 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 19:15:54.525042 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525029 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 19:15:54.525097 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525048 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 19:15:54.525097 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525057 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 19:15:54.525097 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525065 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 19:15:54.525097 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525077 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 19:15:54.525097 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525085 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 19:15:54.525676 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525665 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 19:15:54.525722 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525678 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 19:15:54.525722 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525689 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 19:15:54.525722 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525698 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 19:15:54.525722 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525715 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 19:15:54.525852 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.525727 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 19:15:54.526616 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.526606 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 19:15:54.526669 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.526618 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 19:15:54.529718 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.529703 2562 server.go:1268] "Started kubelet" Jul 15 19:15:54.529838 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.529803 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 19:15:54.529940 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.529868 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 19:15:54.529999 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.529874 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 19:15:54.534501 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.534478 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 19:15:54.534854 ip-10-0-140-236 systemd[1]: Started Kubernetes Kubelet. Jul 15 19:15:54.536275 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.536253 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 15 19:15:54.536368 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.536346 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fqc9z" Jul 15 19:15:54.537355 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.537331 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 19:15:54.537355 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.537337 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-236.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 19:15:54.540681 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.540654 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 19:15:54.540753 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.540722 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 19:15:54.541271 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.540421 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-236.ec2.internal.18c28c907941e622 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-236.ec2.internal,UID:ip-10-0-140-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-236.ec2.internal,},FirstTimestamp:2026-07-15 19:15:54.529678882 +0000 UTC m=+0.366383651,LastTimestamp:2026-07-15 19:15:54.529678882 +0000 UTC m=+0.366383651,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-236.ec2.internal,}" Jul 15 19:15:54.541369 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.541355 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fqc9z" Jul 15 19:15:54.542001 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.541979 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 19:15:54.542081 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.541982 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 19:15:54.542153 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.542133 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:54.542765 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.542746 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 19:15:54.542923 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.542745 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 19:15:54.542923 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.542923 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 19:15:54.543063 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543036 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 19:15:54.543063 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543047 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 15 19:15:54.543063 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543055 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 15 19:15:54.543063 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543048 2562 factory.go:55] Registering systemd factory Jul 15 19:15:54.543255 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543074 2562 factory.go:223] Registration of the systemd container factory successfully Jul 15 19:15:54.543383 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543369 2562 factory.go:153] Registering CRI-O factory Jul 15 19:15:54.543499 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543482 2562 factory.go:223] Registration of the crio container factory successfully Jul 15 19:15:54.543563 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543509 2562 factory.go:103] Registering Raw factory Jul 15 19:15:54.543563 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543521 2562 manager.go:1196] Started watching for new ooms in manager Jul 15 19:15:54.543975 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.543965 2562 manager.go:349] Starting recovery of all containers Jul 15 19:15:54.547250 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.547222 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 19:15:54.549866 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.549848 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:15:54.554366 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.554206 2562 manager.go:354] Recovery completed Jul 15 19:15:54.555681 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.555649 2562 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 15 19:15:54.558595 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.558567 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:15:54.560647 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.560570 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:15:54.560709 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.560652 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:15:54.560709 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.560661 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:15:54.561244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.561229 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 19:15:54.561244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.561242 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:15:54.561244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.561245 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 19:15:54.561382 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.561264 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:15:54.562389 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.562369 2562 policy_none.go:49] "None policy: Start" Jul 15 19:15:54.562475 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.562414 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 19:15:54.562475 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.562431 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 19:15:54.562570 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.562519 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:15:54.562570 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.562550 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:15:54.562570 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.562564 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:15:54.563984 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.563842 2562 policy_none.go:47] "Start" Jul 15 19:15:54.592144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.592120 2562 manager.go:337] "Starting Device Plugin manager" Jul 15 19:15:54.592237 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.592164 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 19:15:54.592237 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.592177 2562 server.go:87] "Starting device plugin registration server" Jul 15 19:15:54.592435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.592421 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 19:15:54.592499 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.592437 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 19:15:54.592565 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.592547 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 19:15:54.592677 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.592666 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 19:15:54.592677 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.592676 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 19:15:54.593180 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.593161 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 19:15:54.593252 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.593211 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:54.692861 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.692826 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:15:54.694072 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.693927 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:15:54.694072 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.693972 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:15:54.694072 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.693988 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:15:54.694072 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.694015 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.698863 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.698843 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.698959 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.698878 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-236.ec2.internal\": node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:54.713265 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.713201 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:54.735354 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.735317 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 19:15:54.736648 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.736627 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 19:15:54.736710 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.736665 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 19:15:54.736710 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.736689 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 19:15:54.736795 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.736724 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 19:15:54.738557 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.738534 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:15:54.813861 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.813821 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:54.837005 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.836962 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal"] Jul 15 19:15:54.837085 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.837055 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:15:54.838636 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.838621 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:15:54.838700 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.838653 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:15:54.838700 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.838665 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:15:54.839963 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.839951 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:15:54.840119 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.840091 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.840154 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.840127 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:15:54.840777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.840762 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:15:54.840855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.840790 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:15:54.840855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.840803 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:15:54.840855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.840820 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:15:54.840855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.840840 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:15:54.840855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.840853 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:15:54.842126 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.842111 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.842169 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.842143 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:15:54.842927 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.842912 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:15:54.842999 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.842936 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:15:54.842999 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.842945 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:15:54.845256 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.845237 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b50a42c21712f0bb6adb1e8ff74aa182-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"b50a42c21712f0bb6adb1e8ff74aa182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.845324 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.845272 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b50a42c21712f0bb6adb1e8ff74aa182-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"b50a42c21712f0bb6adb1e8ff74aa182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.845324 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.845300 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df1b6af21dffcdeba9447749a6e14c99-config\") pod \"kube-apiserver-proxy-ip-10-0-140-236.ec2.internal\" (UID: \"df1b6af21dffcdeba9447749a6e14c99\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.857167 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.857147 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-236.ec2.internal\" not found" node="ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.861534 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.861518 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-236.ec2.internal\" not found" node="ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.914147 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:54.914119 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:54.946449 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.946418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b50a42c21712f0bb6adb1e8ff74aa182-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"b50a42c21712f0bb6adb1e8ff74aa182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.946523 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.946450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b50a42c21712f0bb6adb1e8ff74aa182-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"b50a42c21712f0bb6adb1e8ff74aa182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.946523 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.946470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df1b6af21dffcdeba9447749a6e14c99-config\") pod \"kube-apiserver-proxy-ip-10-0-140-236.ec2.internal\" (UID: \"df1b6af21dffcdeba9447749a6e14c99\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.946612 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.946521 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df1b6af21dffcdeba9447749a6e14c99-config\") pod \"kube-apiserver-proxy-ip-10-0-140-236.ec2.internal\" (UID: \"df1b6af21dffcdeba9447749a6e14c99\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.946612 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.946531 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b50a42c21712f0bb6adb1e8ff74aa182-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"b50a42c21712f0bb6adb1e8ff74aa182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" Jul 15 19:15:54.946612 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:54.946521 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b50a42c21712f0bb6adb1e8ff74aa182-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal\" (UID: \"b50a42c21712f0bb6adb1e8ff74aa182\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" Jul 15 19:15:55.014772 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:55.014708 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:55.115409 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:55.115366 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:55.158917 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.158884 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" Jul 15 19:15:55.165110 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.165090 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" Jul 15 19:15:55.216232 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:55.216195 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:55.316733 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:55.316655 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:55.417115 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:55.417090 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:55.479688 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.479658 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 19:15:55.480163 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.479795 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 19:15:55.480163 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.479820 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 19:15:55.517989 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:55.517950 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:55.546512 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.546467 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 19:10:54 +0000 UTC" deadline="2028-03-13 01:51:04.715556585 +0000 UTC" Jul 15 19:15:55.546512 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.546510 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14550h35m9.169051602s" Jul 15 19:15:55.618821 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:55.618750 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:55.663053 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:55.663012 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50a42c21712f0bb6adb1e8ff74aa182.slice/crio-1a16cbfceb86156cf1643a3a4aacf3ae556716ca703b2b8d6831ed06ef201dae WatchSource:0}: Error finding container 1a16cbfceb86156cf1643a3a4aacf3ae556716ca703b2b8d6831ed06ef201dae: Status 404 returned error can't find the container with id 1a16cbfceb86156cf1643a3a4aacf3ae556716ca703b2b8d6831ed06ef201dae Jul 15 19:15:55.663262 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:55.663238 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf1b6af21dffcdeba9447749a6e14c99.slice/crio-837a994f47cfb5aa32eb15a4aebd945f56a868cab5bdb76feffeacca02794366 WatchSource:0}: Error finding container 837a994f47cfb5aa32eb15a4aebd945f56a868cab5bdb76feffeacca02794366: Status 404 returned error can't find the container with id 837a994f47cfb5aa32eb15a4aebd945f56a868cab5bdb76feffeacca02794366 Jul 15 19:15:55.668328 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.668315 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:15:55.719195 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:55.719165 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:55.740492 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.740445 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" event={"ID":"b50a42c21712f0bb6adb1e8ff74aa182","Type":"ContainerStarted","Data":"1a16cbfceb86156cf1643a3a4aacf3ae556716ca703b2b8d6831ed06ef201dae"} Jul 15 19:15:55.741459 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.741435 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" event={"ID":"df1b6af21dffcdeba9447749a6e14c99","Type":"ContainerStarted","Data":"837a994f47cfb5aa32eb15a4aebd945f56a868cab5bdb76feffeacca02794366"} Jul 15 19:15:55.819898 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:15:55.819863 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-236.ec2.internal\" not found" Jul 15 19:15:55.850395 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.850369 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:15:55.861240 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.861219 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:15:55.943459 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.943370 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" Jul 15 19:15:55.957586 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.957554 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 19:15:55.958462 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:55.958447 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" Jul 15 19:15:56.057114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.057079 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 19:15:56.520716 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.520684 2562 apiserver.go:52] "Watching apiserver" Jul 15 19:15:56.526544 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.526516 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 19:15:56.526884 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.526866 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt","openshift-cluster-node-tuning-operator/tuned-nxt6n","openshift-dns/node-resolver-bcrrg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal","openshift-network-operator/iptables-alerter-gt4m9","kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal"] Jul 15 19:15:56.528258 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.528232 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.529875 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.529854 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.530898 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.530877 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.530995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.530925 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 19:15:56.531408 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.531392 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 19:15:56.531497 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.531424 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 19:15:56.531497 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.531433 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j4zxk\"" Jul 15 19:15:56.531994 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.531976 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.532356 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.532340 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 19:15:56.532512 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.532490 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jm2fs\"" Jul 15 19:15:56.532675 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.532522 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:15:56.533502 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.533484 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ptc52\"" Jul 15 19:15:56.533502 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.533494 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 19:15:56.533661 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.533494 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 19:15:56.534638 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.534562 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 19:15:56.534754 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.534735 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 19:15:56.534850 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.534723 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:15:56.534850 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.534827 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h57sb\"" Jul 15 19:15:56.543344 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.543324 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 19:15:56.543633 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.543616 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-host\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9504f5-8c66-4ec0-a5c3-3406952b5302-tmp-dir\") pod \"node-resolver-bcrrg\" (UID: \"0a9504f5-8c66-4ec0-a5c3-3406952b5302\") " pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-socket-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-systemd\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9459af9e-ce63-4184-8cdf-3c41c6dee8ac-host-slash\") pod \"iptables-alerter-gt4m9\" (UID: \"9459af9e-ce63-4184-8cdf-3c41c6dee8ac\") " pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555743 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfsz2\" (UniqueName: \"kubernetes.io/projected/9459af9e-ce63-4184-8cdf-3c41c6dee8ac-kube-api-access-lfsz2\") pod \"iptables-alerter-gt4m9\" (UID: \"9459af9e-ce63-4184-8cdf-3c41c6dee8ac\") " pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-sysconfig\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555822 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-kubernetes\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-tuned\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-registration-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-sys\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a9504f5-8c66-4ec0-a5c3-3406952b5302-hosts-file\") pod \"node-resolver-bcrrg\" (UID: \"0a9504f5-8c66-4ec0-a5c3-3406952b5302\") " pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8567f5d-d256-42ed-bb37-7bf7727f4882-tmp\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9459af9e-ce63-4184-8cdf-3c41c6dee8ac-iptables-alerter-script\") pod \"iptables-alerter-gt4m9\" (UID: \"9459af9e-ce63-4184-8cdf-3c41c6dee8ac\") " pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.555985 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.556233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556007 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-device-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556030 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-sys-fs\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556070 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm8fp\" (UniqueName: \"kubernetes.io/projected/caababca-b2f4-438e-aadd-0a5c7c8576fb-kube-api-access-mm8fp\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556091 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-modprobe-d\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556113 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-sysctl-d\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556136 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-run\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556171 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556196 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-sysctl-conf\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-var-lib-kubelet\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4df9m\" (UniqueName: \"kubernetes.io/projected/d8567f5d-d256-42ed-bb37-7bf7727f4882-kube-api-access-4df9m\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-lib-modules\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.556936 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.556624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tbdg\" (UniqueName: \"kubernetes.io/projected/0a9504f5-8c66-4ec0-a5c3-3406952b5302-kube-api-access-9tbdg\") pod \"node-resolver-bcrrg\" (UID: \"0a9504f5-8c66-4ec0-a5c3-3406952b5302\") " pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.557445 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.557420 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:15:56.581818 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.581789 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jzxzr" Jul 15 19:15:56.609073 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.609043 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:15:56.657613 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657552 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8567f5d-d256-42ed-bb37-7bf7727f4882-tmp\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.657613 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657615 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9459af9e-ce63-4184-8cdf-3c41c6dee8ac-iptables-alerter-script\") pod \"iptables-alerter-gt4m9\" (UID: \"9459af9e-ce63-4184-8cdf-3c41c6dee8ac\") " pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657643 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-device-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-sys-fs\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657715 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm8fp\" (UniqueName: \"kubernetes.io/projected/caababca-b2f4-438e-aadd-0a5c7c8576fb-kube-api-access-mm8fp\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657738 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-modprobe-d\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-sysctl-d\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657763 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-run\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657793 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-device-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-sysctl-conf\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657856 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-var-lib-kubelet\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.657897 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657896 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4df9m\" (UniqueName: \"kubernetes.io/projected/d8567f5d-d256-42ed-bb37-7bf7727f4882-kube-api-access-4df9m\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657905 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-modprobe-d\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-lib-modules\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657793 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-sys-fs\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tbdg\" (UniqueName: \"kubernetes.io/projected/0a9504f5-8c66-4ec0-a5c3-3406952b5302-kube-api-access-9tbdg\") pod \"node-resolver-bcrrg\" (UID: \"0a9504f5-8c66-4ec0-a5c3-3406952b5302\") " pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657968 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-run\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657972 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658026 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-sysctl-conf\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658057 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657905 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-sysctl-d\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658118 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-var-lib-kubelet\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658125 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-lib-modules\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.657981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-host\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658152 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-host\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9504f5-8c66-4ec0-a5c3-3406952b5302-tmp-dir\") pod \"node-resolver-bcrrg\" (UID: \"0a9504f5-8c66-4ec0-a5c3-3406952b5302\") " pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-socket-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-systemd\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9459af9e-ce63-4184-8cdf-3c41c6dee8ac-host-slash\") pod \"iptables-alerter-gt4m9\" (UID: \"9459af9e-ce63-4184-8cdf-3c41c6dee8ac\") " pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.658465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658221 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfsz2\" (UniqueName: \"kubernetes.io/projected/9459af9e-ce63-4184-8cdf-3c41c6dee8ac-kube-api-access-lfsz2\") pod \"iptables-alerter-gt4m9\" (UID: \"9459af9e-ce63-4184-8cdf-3c41c6dee8ac\") " pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658226 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9459af9e-ce63-4184-8cdf-3c41c6dee8ac-iptables-alerter-script\") pod \"iptables-alerter-gt4m9\" (UID: \"9459af9e-ce63-4184-8cdf-3c41c6dee8ac\") " pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658244 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-sysconfig\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658265 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-kubernetes\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658265 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-systemd\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-tuned\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658296 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-registration-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-sys\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658313 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-socket-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a9504f5-8c66-4ec0-a5c3-3406952b5302-hosts-file\") pod \"node-resolver-bcrrg\" (UID: \"0a9504f5-8c66-4ec0-a5c3-3406952b5302\") " pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658378 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-sysconfig\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658415 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-kubernetes\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658429 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8567f5d-d256-42ed-bb37-7bf7727f4882-sys\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658482 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0a9504f5-8c66-4ec0-a5c3-3406952b5302-hosts-file\") pod \"node-resolver-bcrrg\" (UID: \"0a9504f5-8c66-4ec0-a5c3-3406952b5302\") " pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9459af9e-ce63-4184-8cdf-3c41c6dee8ac-host-slash\") pod \"iptables-alerter-gt4m9\" (UID: \"9459af9e-ce63-4184-8cdf-3c41c6dee8ac\") " pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658522 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/caababca-b2f4-438e-aadd-0a5c7c8576fb-registration-dir\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.658995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.658536 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9504f5-8c66-4ec0-a5c3-3406952b5302-tmp-dir\") pod \"node-resolver-bcrrg\" (UID: \"0a9504f5-8c66-4ec0-a5c3-3406952b5302\") " pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.661288 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.661260 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8567f5d-d256-42ed-bb37-7bf7727f4882-etc-tuned\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.661412 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.661315 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8567f5d-d256-42ed-bb37-7bf7727f4882-tmp\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.666782 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.666762 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm8fp\" (UniqueName: \"kubernetes.io/projected/caababca-b2f4-438e-aadd-0a5c7c8576fb-kube-api-access-mm8fp\") pod \"aws-ebs-csi-driver-node-7gqrt\" (UID: \"caababca-b2f4-438e-aadd-0a5c7c8576fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.667256 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.667167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4df9m\" (UniqueName: \"kubernetes.io/projected/d8567f5d-d256-42ed-bb37-7bf7727f4882-kube-api-access-4df9m\") pod \"tuned-nxt6n\" (UID: \"d8567f5d-d256-42ed-bb37-7bf7727f4882\") " pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.667256 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.667168 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfsz2\" (UniqueName: \"kubernetes.io/projected/9459af9e-ce63-4184-8cdf-3c41c6dee8ac-kube-api-access-lfsz2\") pod \"iptables-alerter-gt4m9\" (UID: \"9459af9e-ce63-4184-8cdf-3c41c6dee8ac\") " pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.667531 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.667511 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tbdg\" (UniqueName: \"kubernetes.io/projected/0a9504f5-8c66-4ec0-a5c3-3406952b5302-kube-api-access-9tbdg\") pod \"node-resolver-bcrrg\" (UID: \"0a9504f5-8c66-4ec0-a5c3-3406952b5302\") " pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.839822 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.839744 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" Jul 15 19:15:56.845705 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.845682 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" Jul 15 19:15:56.862440 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.862407 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bcrrg" Jul 15 19:15:56.865368 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.865339 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gt4m9" Jul 15 19:15:56.909006 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:56.908971 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:15:57.440321 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:57.440291 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8567f5d_d256_42ed_bb37_7bf7727f4882.slice/crio-fef0091d1dd01640aa042a24ca0152ac5763fdbadbf1661657f2f75111fd85b0 WatchSource:0}: Error finding container fef0091d1dd01640aa042a24ca0152ac5763fdbadbf1661657f2f75111fd85b0: Status 404 returned error can't find the container with id fef0091d1dd01640aa042a24ca0152ac5763fdbadbf1661657f2f75111fd85b0 Jul 15 19:15:57.441375 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:57.441347 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaababca_b2f4_438e_aadd_0a5c7c8576fb.slice/crio-e7da3e7145adc59053b8dfda0a10a4ac2e7f49aaf3c32c665eaddffa057021da WatchSource:0}: Error finding container e7da3e7145adc59053b8dfda0a10a4ac2e7f49aaf3c32c665eaddffa057021da: Status 404 returned error can't find the container with id e7da3e7145adc59053b8dfda0a10a4ac2e7f49aaf3c32c665eaddffa057021da Jul 15 19:15:57.443182 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:57.443024 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a9504f5_8c66_4ec0_a5c3_3406952b5302.slice/crio-b5f214f0610be07468958206e601ea0656e473d636d90e88536333755bb92b99 WatchSource:0}: Error finding container b5f214f0610be07468958206e601ea0656e473d636d90e88536333755bb92b99: Status 404 returned error can't find the container with id b5f214f0610be07468958206e601ea0656e473d636d90e88536333755bb92b99 Jul 15 19:15:57.444155 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:15:57.444001 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9459af9e_ce63_4184_8cdf_3c41c6dee8ac.slice/crio-487784868b9667be14a9a4df84ccc85f32bfdfd7d2edb1f7d525e3fbbb54465f WatchSource:0}: Error finding container 487784868b9667be14a9a4df84ccc85f32bfdfd7d2edb1f7d525e3fbbb54465f: Status 404 returned error can't find the container with id 487784868b9667be14a9a4df84ccc85f32bfdfd7d2edb1f7d525e3fbbb54465f Jul 15 19:15:57.746730 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:57.746410 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" event={"ID":"df1b6af21dffcdeba9447749a6e14c99","Type":"ContainerStarted","Data":"57304734e65a1d02fcf3799903cba449a0aad67f53108bb23ca2ea28ebac4b2b"} Jul 15 19:15:57.747618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:57.747595 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gt4m9" event={"ID":"9459af9e-ce63-4184-8cdf-3c41c6dee8ac","Type":"ContainerStarted","Data":"487784868b9667be14a9a4df84ccc85f32bfdfd7d2edb1f7d525e3fbbb54465f"} Jul 15 19:15:57.748642 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:57.748613 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bcrrg" event={"ID":"0a9504f5-8c66-4ec0-a5c3-3406952b5302","Type":"ContainerStarted","Data":"b5f214f0610be07468958206e601ea0656e473d636d90e88536333755bb92b99"} Jul 15 19:15:57.749651 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:57.749630 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" event={"ID":"d8567f5d-d256-42ed-bb37-7bf7727f4882","Type":"ContainerStarted","Data":"fef0091d1dd01640aa042a24ca0152ac5763fdbadbf1661657f2f75111fd85b0"} Jul 15 19:15:57.750632 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:57.750614 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" event={"ID":"caababca-b2f4-438e-aadd-0a5c7c8576fb","Type":"ContainerStarted","Data":"e7da3e7145adc59053b8dfda0a10a4ac2e7f49aaf3c32c665eaddffa057021da"} Jul 15 19:15:57.751835 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:57.751815 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" event={"ID":"b50a42c21712f0bb6adb1e8ff74aa182","Type":"ContainerStarted","Data":"7df1c4e34b42c0cab7e17bf3729123197c4e0b13f0256e11733637817719cf10"} Jul 15 19:15:57.774999 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:15:57.774954 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-236.ec2.internal" podStartSLOduration=1.774940577 podStartE2EDuration="1.774940577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:15:57.76068657 +0000 UTC m=+3.597391350" watchObservedRunningTime="2026-07-15 19:15:57.774940577 +0000 UTC m=+3.611645344" Jul 15 19:16:02.762035 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:02.761997 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bcrrg" event={"ID":"0a9504f5-8c66-4ec0-a5c3-3406952b5302","Type":"ContainerStarted","Data":"38f02af3a52b578c207cf79ad00ccd19c0bf54af398a073a56dfc2ee77f08361"} Jul 15 19:16:02.763803 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:02.763766 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" event={"ID":"d8567f5d-d256-42ed-bb37-7bf7727f4882","Type":"ContainerStarted","Data":"56cceecbb955e3c9cb07b99d71a86dc99a6cab1b9274e1497f2a347b61948fc0"} Jul 15 19:16:02.765468 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:02.765444 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" event={"ID":"caababca-b2f4-438e-aadd-0a5c7c8576fb","Type":"ContainerStarted","Data":"9d9b77360b5d6975b8053fe7cb7a81ad64c152715f84ad166adec8752957f76f"} Jul 15 19:16:02.776874 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:02.776832 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bcrrg" podStartSLOduration=3.991282459 podStartE2EDuration="8.776819161s" totalImagesPullingTime="4.785536702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:02.776533479 +0000 UTC m=+8.613238258" watchObservedRunningTime="2026-07-15 19:16:02.776819161 +0000 UTC m=+8.613523937" Jul 15 19:16:02.791724 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:02.791670 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nxt6n" podStartSLOduration=3.99339992 podStartE2EDuration="8.791651579s" totalImagesPullingTime="4.798251659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:02.791535827 +0000 UTC m=+8.628240600" watchObservedRunningTime="2026-07-15 19:16:02.791651579 +0000 UTC m=+8.628356357" Jul 15 19:16:03.201237 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:03.201207 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 19:16:03.598837 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:03.598722 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T19:16:03.201223643Z","UUID":"897dea2f-015a-4d8c-8996-5fced0d7a2e2","Handler":null,"Name":"","Endpoint":""} Jul 15 19:16:03.600899 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:03.600873 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 19:16:03.601048 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:03.600920 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 19:16:03.769150 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:03.769063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gt4m9" event={"ID":"9459af9e-ce63-4184-8cdf-3c41c6dee8ac","Type":"ContainerStarted","Data":"47c7c145bb21201fa46d463615f147ec73c1649b8422f24ffa2c4e9dc985813b"} Jul 15 19:16:03.771102 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:03.771063 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" event={"ID":"caababca-b2f4-438e-aadd-0a5c7c8576fb","Type":"ContainerStarted","Data":"0d8b97800d94966723b41c37d240eb7d915c73da3990135b7046ea399e5062b4"} Jul 15 19:16:04.774882 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:04.774849 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" event={"ID":"caababca-b2f4-438e-aadd-0a5c7c8576fb","Type":"ContainerStarted","Data":"827b734170d688a98a21284575deec725d42953a7dd815e43501f9f167bbd64b"} Jul 15 19:16:04.792563 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:04.792521 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7gqrt" podStartSLOduration=4.547404496 podStartE2EDuration="10.792506696s" totalImagesPullingTime="6.2451022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:04.792445699 +0000 UTC m=+10.629150487" watchObservedRunningTime="2026-07-15 19:16:04.792506696 +0000 UTC m=+10.629211473" Jul 15 19:16:04.792768 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:04.792748 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gt4m9" podStartSLOduration=6.005798627 podStartE2EDuration="10.792741064s" totalImagesPullingTime="4.786942437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:03.783986611 +0000 UTC m=+9.620691387" watchObservedRunningTime="2026-07-15 19:16:04.792741064 +0000 UTC m=+10.629445840" Jul 15 19:16:08.782990 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:08.782839 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal_b50a42c21712f0bb6adb1e8ff74aa182/setup/0.log" Jul 15 19:16:08.783476 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:08.783003 2562 generic.go:356] "Generic (PLEG): container finished" podID="b50a42c21712f0bb6adb1e8ff74aa182" containerID="7df1c4e34b42c0cab7e17bf3729123197c4e0b13f0256e11733637817719cf10" exitCode=1 Jul 15 19:16:08.783476 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:08.783038 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" event={"ID":"b50a42c21712f0bb6adb1e8ff74aa182","Type":"ContainerDied","Data":"7df1c4e34b42c0cab7e17bf3729123197c4e0b13f0256e11733637817719cf10"} Jul 15 19:16:09.785982 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:09.785953 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal_b50a42c21712f0bb6adb1e8ff74aa182/setup/0.log" Jul 15 19:16:09.786438 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:09.786015 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" event={"ID":"b50a42c21712f0bb6adb1e8ff74aa182","Type":"ContainerStarted","Data":"fd6cb4bbfe545582c647d6559a7b1d70e0775f6c2cec99a9238f307e19da1287"} Jul 15 19:16:17.440207 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.440180 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jzxzr" Jul 15 19:16:17.500366 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.500339 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-vxc2j"] Jul 15 19:16:17.502740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.502725 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-j8jpj"] Jul 15 19:16:17.502860 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.502843 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:17.505060 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.505039 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:17.505176 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.505100 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:17.505414 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.505401 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 19:16:17.505637 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.505616 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 19:16:17.506614 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.506599 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wgck6\"" Jul 15 19:16:17.552609 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.552566 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-z6fkb"] Jul 15 19:16:17.555076 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.555056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.558043 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.558023 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 19:16:17.558043 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.558041 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 19:16:17.558181 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.558099 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 19:16:17.558668 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.558654 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sb9tm\"" Jul 15 19:16:17.574606 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.574589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:17.574688 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.574618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a87c2704-0fe6-4050-87d0-cc80a8da963b-agent-certs\") pod \"konnectivity-agent-vxc2j\" (UID: \"a87c2704-0fe6-4050-87d0-cc80a8da963b\") " pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:17.574688 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.574638 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/648cd87d-755f-4648-97d3-0857e63a32bb-dbus\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:17.574688 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.574654 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/648cd87d-755f-4648-97d3-0857e63a32bb-kubelet-config\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:17.574688 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.574672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a87c2704-0fe6-4050-87d0-cc80a8da963b-konnectivity-ca\") pod \"konnectivity-agent-vxc2j\" (UID: \"a87c2704-0fe6-4050-87d0-cc80a8da963b\") " pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:17.591098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.591070 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-btwt9"] Jul 15 19:16:17.593633 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.593616 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.598768 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.598733 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"multus-btwt9\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-multus\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="7b0824c5-d7a8-4c30-91f5-f2b60f034e4d" pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.598875 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.598851 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"cni-copy-resources\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-multus\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" type="*v1.ConfigMap" Jul 15 19:16:17.599085 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.599068 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-cdt4d"] Jul 15 19:16:17.599847 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.599826 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-multus\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 15 19:16:17.600639 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.600619 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-multus\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 15 19:16:17.601784 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.601768 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.604812 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.604790 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kj4zw\"" Jul 15 19:16:17.604890 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.604854 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 19:16:17.604890 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.604876 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 19:16:17.612093 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.612072 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"multus-daemon-config\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-multus\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" type="*v1.ConfigMap" Jul 15 19:16:17.612183 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.612166 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-ncmhl\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-multus\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-multus\"/\"default-dockercfg-ncmhl\"" type="*v1.Secret" Jul 15 19:16:17.629649 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.629627 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-gtxhr"] Jul 15 19:16:17.632164 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.632150 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:17.632221 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.632207 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:17.674943 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.674918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-cnibin\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.674943 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.674944 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-run-multus-certs\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.674962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-cni-binary-copy\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.674977 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-hostroot\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.674992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcxjr\" (UniqueName: \"kubernetes.io/projected/b087f738-59f9-4bcb-b227-33cfdee6adc9-kube-api-access-jcxjr\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:17.675144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675038 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/648cd87d-755f-4648-97d3-0857e63a32bb-dbus\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:17.675144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675054 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-run-k8s-cni-cncf-io\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/648cd87d-755f-4648-97d3-0857e63a32bb-kubelet-config\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:17.675144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-var-lib-cni-bin\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675144 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/648cd87d-755f-4648-97d3-0857e63a32bb-kubelet-config\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/320343bc-1b3b-4ca7-9ae4-1e073451be95-cni-binary-copy\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675223 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/648cd87d-755f-4648-97d3-0857e63a32bb-dbus\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-os-release\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-conf-dir\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675273 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/320343bc-1b3b-4ca7-9ae4-1e073451be95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a87c2704-0fe6-4050-87d0-cc80a8da963b-agent-certs\") pod \"konnectivity-agent-vxc2j\" (UID: \"a87c2704-0fe6-4050-87d0-cc80a8da963b\") " pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675335 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-var-lib-cni-multus\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-system-cni-dir\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-socket-dir-parent\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675379 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-system-cni-dir\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.675397 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:17.675435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/320343bc-1b3b-4ca7-9ae4-1e073451be95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675491 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.675532 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret podName:648cd87d-755f-4648-97d3-0857e63a32bb nodeName:}" failed. No retries permitted until 2026-07-15 19:16:18.17551727 +0000 UTC m=+24.012222026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret") pod "global-pull-secret-syncer-j8jpj" (UID: "648cd87d-755f-4648-97d3-0857e63a32bb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7f85\" (UniqueName: \"kubernetes.io/projected/aa06f283-9e57-4543-a309-5605c58b0893-kube-api-access-v7f85\") pod \"node-ca-z6fkb\" (UID: \"aa06f283-9e57-4543-a309-5605c58b0893\") " pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675634 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-etc-kubernetes\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-run-netns\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-var-lib-kubelet\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x2cd\" (UniqueName: \"kubernetes.io/projected/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-kube-api-access-5x2cd\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-cnibin\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675742 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aa06f283-9e57-4543-a309-5605c58b0893-serviceca\") pod \"node-ca-z6fkb\" (UID: \"aa06f283-9e57-4543-a309-5605c58b0893\") " pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675771 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-cni-dir\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a87c2704-0fe6-4050-87d0-cc80a8da963b-konnectivity-ca\") pod \"konnectivity-agent-vxc2j\" (UID: \"a87c2704-0fe6-4050-87d0-cc80a8da963b\") " pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675814 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa06f283-9e57-4543-a309-5605c58b0893-host\") pod \"node-ca-z6fkb\" (UID: \"aa06f283-9e57-4543-a309-5605c58b0893\") " pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-daemon-config\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-os-release\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.675902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.675878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq4bj\" (UniqueName: \"kubernetes.io/projected/320343bc-1b3b-4ca7-9ae4-1e073451be95-kube-api-access-fq4bj\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.676370 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.676190 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a87c2704-0fe6-4050-87d0-cc80a8da963b-konnectivity-ca\") pod \"konnectivity-agent-vxc2j\" (UID: \"a87c2704-0fe6-4050-87d0-cc80a8da963b\") " pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:17.679447 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.679430 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a87c2704-0fe6-4050-87d0-cc80a8da963b-agent-certs\") pod \"konnectivity-agent-vxc2j\" (UID: \"a87c2704-0fe6-4050-87d0-cc80a8da963b\") " pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:17.702181 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.702126 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-ftxlr"] Jul 15 19:16:17.704624 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.704610 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:17.704681 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.704666 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:17.733929 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.733902 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-9gs6l"] Jul 15 19:16:17.736813 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.736797 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.740717 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.740697 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 19:16:17.740813 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.740767 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 19:16:17.740870 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.740850 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 19:16:17.741033 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.741020 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 19:16:17.742019 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.742002 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 19:16:17.742100 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.742036 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 19:16:17.742160 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.742115 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p5gk7\"" Jul 15 19:16:17.776852 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-cnibin\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.776852 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-run-multus-certs\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-run-ovn-kubernetes\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gxrl\" (UniqueName: \"kubernetes.io/projected/b1d40045-d2ca-4b74-8192-248a35efdf7d-kube-api-access-9gxrl\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-cni-binary-copy\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-hostroot\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776936 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1d40045-d2ca-4b74-8192-248a35efdf7d-ovnkube-script-lib\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcxjr\" (UniqueName: \"kubernetes.io/projected/b087f738-59f9-4bcb-b227-33cfdee6adc9-kube-api-access-jcxjr\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-run-multus-certs\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-cni-netd\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.776962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-cnibin\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-run-k8s-cni-cncf-io\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777006 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-hostroot\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777036 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777071 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-run-k8s-cni-cncf-io\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-var-lib-cni-bin\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/320343bc-1b3b-4ca7-9ae4-1e073451be95-cni-binary-copy\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777155 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-var-lib-cni-bin\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777157 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-run-ovn\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777196 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-os-release\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-conf-dir\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777228 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1d40045-d2ca-4b74-8192-248a35efdf7d-ovnkube-config\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/320343bc-1b3b-4ca7-9ae4-1e073451be95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777273 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-conf-dir\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-var-lib-cni-multus\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-slash\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777331 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-var-lib-cni-multus\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777341 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-os-release\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-system-cni-dir\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.777464 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-socket-dir-parent\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-system-cni-dir\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777403 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/320343bc-1b3b-4ca7-9ae4-1e073451be95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777422 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-node-log\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777453 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-system-cni-dir\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777385 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-system-cni-dir\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777483 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-socket-dir-parent\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-run-systemd\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1d40045-d2ca-4b74-8192-248a35efdf7d-ovn-node-metrics-cert\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777529 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7f85\" (UniqueName: \"kubernetes.io/projected/aa06f283-9e57-4543-a309-5605c58b0893-kube-api-access-v7f85\") pod \"node-ca-z6fkb\" (UID: \"aa06f283-9e57-4543-a309-5605c58b0893\") " pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777546 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-etc-kubernetes\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777570 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-kubelet\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777604 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-systemd-units\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-log-socket\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777633 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777648 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-run-netns\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777642 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-etc-kubernetes\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777680 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-run-netns\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777686 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-var-lib-kubelet\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x2cd\" (UniqueName: \"kubernetes.io/projected/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-kube-api-access-5x2cd\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777742 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-host-var-lib-kubelet\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-cnibin\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777835 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/320343bc-1b3b-4ca7-9ae4-1e073451be95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-etc-openvswitch\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777868 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/320343bc-1b3b-4ca7-9ae4-1e073451be95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.777887 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-cnibin\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.777934 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.777995 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs podName:b087f738-59f9-4bcb-b227-33cfdee6adc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:18.277979662 +0000 UTC m=+24.114684417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs") pod "network-metrics-daemon-gtxhr" (UID: "b087f738-59f9-4bcb-b227-33cfdee6adc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778012 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aa06f283-9e57-4543-a309-5605c58b0893-serviceca\") pod \"node-ca-z6fkb\" (UID: \"aa06f283-9e57-4543-a309-5605c58b0893\") " pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778105 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-cni-dir\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778133 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa06f283-9e57-4543-a309-5605c58b0893-host\") pod \"node-ca-z6fkb\" (UID: \"aa06f283-9e57-4543-a309-5605c58b0893\") " pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778171 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aa06f283-9e57-4543-a309-5605c58b0893-host\") pod \"node-ca-z6fkb\" (UID: \"aa06f283-9e57-4543-a309-5605c58b0893\") " pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.778618 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778187 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-cni-dir\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-daemon-config\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778234 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-os-release\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778259 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778282 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-run-netns\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778307 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-var-lib-openvswitch\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778310 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/320343bc-1b3b-4ca7-9ae4-1e073451be95-os-release\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778342 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-run-openvswitch\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778360 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-cni-bin\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778373 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1d40045-d2ca-4b74-8192-248a35efdf7d-env-overrides\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq4bj\" (UniqueName: \"kubernetes.io/projected/320343bc-1b3b-4ca7-9ae4-1e073451be95-kube-api-access-fq4bj\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:17.779074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.778482 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/aa06f283-9e57-4543-a309-5605c58b0893-serviceca\") pod \"node-ca-z6fkb\" (UID: \"aa06f283-9e57-4543-a309-5605c58b0893\") " pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.795728 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.795704 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7f85\" (UniqueName: \"kubernetes.io/projected/aa06f283-9e57-4543-a309-5605c58b0893-kube-api-access-v7f85\") pod \"node-ca-z6fkb\" (UID: \"aa06f283-9e57-4543-a309-5605c58b0893\") " pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.813872 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.813842 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:17.862144 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:16:17.862000 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda87c2704_0fe6_4050_87d0_cc80a8da963b.slice/crio-29860609b68d766daf50d64ba2f38712f9ff0a5e890073821f46bd0b7e3bfe1e WatchSource:0}: Error finding container 29860609b68d766daf50d64ba2f38712f9ff0a5e890073821f46bd0b7e3bfe1e: Status 404 returned error can't find the container with id 29860609b68d766daf50d64ba2f38712f9ff0a5e890073821f46bd0b7e3bfe1e Jul 15 19:16:17.862830 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.862810 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-z6fkb" Jul 15 19:16:17.870865 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:16:17.870846 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa06f283_9e57_4543_a309_5605c58b0893.slice/crio-1554e62d60ad2f92855483656ca9e39ec25574c925a2b34d13b9e981cf521f7b WatchSource:0}: Error finding container 1554e62d60ad2f92855483656ca9e39ec25574c925a2b34d13b9e981cf521f7b: Status 404 returned error can't find the container with id 1554e62d60ad2f92855483656ca9e39ec25574c925a2b34d13b9e981cf521f7b Jul 15 19:16:17.878776 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878758 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-run-ovn\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.878858 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1d40045-d2ca-4b74-8192-248a35efdf7d-ovnkube-config\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.878858 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-slash\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.878858 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-node-log\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878861 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-run-systemd\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878867 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-run-ovn\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878875 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1d40045-d2ca-4b74-8192-248a35efdf7d-ovn-node-metrics-cert\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878924 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-kubelet\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878945 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-run-systemd\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-systemd-units\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878956 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-node-log\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878981 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-slash\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878987 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-kubelet\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.878992 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-systemd-units\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879013 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-log-socket\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-etc-openvswitch\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879045 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-log-socket\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-etc-openvswitch\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-run-netns\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-var-lib-openvswitch\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879174 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-run-openvswitch\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879196 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-run-netns\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-cni-bin\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1d40045-d2ca-4b74-8192-248a35efdf7d-env-overrides\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879242 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-var-lib-openvswitch\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879268 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-run-ovn-kubernetes\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879269 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-run-openvswitch\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879295 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-cni-bin\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879296 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gxrl\" (UniqueName: \"kubernetes.io/projected/b1d40045-d2ca-4b74-8192-248a35efdf7d-kube-api-access-9gxrl\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879338 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1d40045-d2ca-4b74-8192-248a35efdf7d-ovnkube-script-lib\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-cni-netd\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.879723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.880209 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879402 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-run-ovn-kubernetes\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.880209 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879506 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-cni-netd\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.880209 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879523 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b1d40045-d2ca-4b74-8192-248a35efdf7d-ovnkube-config\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.880209 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879546 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b1d40045-d2ca-4b74-8192-248a35efdf7d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.880209 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.879689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b1d40045-d2ca-4b74-8192-248a35efdf7d-env-overrides\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.880209 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.880167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b1d40045-d2ca-4b74-8192-248a35efdf7d-ovnkube-script-lib\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.881752 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.881732 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b1d40045-d2ca-4b74-8192-248a35efdf7d-ovn-node-metrics-cert\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.888941 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:17.888923 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gxrl\" (UniqueName: \"kubernetes.io/projected/b1d40045-d2ca-4b74-8192-248a35efdf7d-kube-api-access-9gxrl\") pod \"ovnkube-node-9gs6l\" (UID: \"b1d40045-d2ca-4b74-8192-248a35efdf7d\") " pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:17.896220 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.896202 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:16:17.896304 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.896224 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:16:17.896304 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.896237 2562 projected.go:196] Error preparing data for projected volume kube-api-access-k2l25 for pod openshift-network-diagnostics/network-check-target-ftxlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:17.896304 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:17.896301 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25 podName:a6cf0904-b4aa-403b-b630-4281b5bcd835 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:18.396283256 +0000 UTC m=+24.232988025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k2l25" (UniqueName: "kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25") pod "network-check-target-ftxlr" (UID: "a6cf0904-b4aa-403b-b630-4281b5bcd835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:18.045468 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.045375 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:18.053971 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:16:18.053949 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d40045_d2ca_4b74_8192_248a35efdf7d.slice/crio-c617f56d2685e4c580856ee32639809875481c199b647043afa502c44bb9c997 WatchSource:0}: Error finding container c617f56d2685e4c580856ee32639809875481c199b647043afa502c44bb9c997: Status 404 returned error can't find the container with id c617f56d2685e4c580856ee32639809875481c199b647043afa502c44bb9c997 Jul 15 19:16:18.181659 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.181620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:18.181797 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.181762 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:18.181852 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.181819 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret podName:648cd87d-755f-4648-97d3-0857e63a32bb nodeName:}" failed. No retries permitted until 2026-07-15 19:16:19.181805123 +0000 UTC m=+25.018509878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret") pod "global-pull-secret-syncer-j8jpj" (UID: "648cd87d-755f-4648-97d3-0857e63a32bb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:18.282031 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.282003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:18.282155 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.282139 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:18.282210 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.282197 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs podName:b087f738-59f9-4bcb-b227-33cfdee6adc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:19.282182331 +0000 UTC m=+25.118887086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs") pod "network-metrics-daemon-gtxhr" (UID: "b087f738-59f9-4bcb-b227-33cfdee6adc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:18.442987 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.442886 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:11:17 +0000 UTC" deadline="2028-01-12 17:57:05.498499837 +0000 UTC" Jul 15 19:16:18.442987 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.442931 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13102h40m47.055571257s" Jul 15 19:16:18.483400 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.483368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:18.483548 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.483502 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:16:18.483548 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.483518 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:16:18.483548 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.483529 2562 projected.go:196] Error preparing data for projected volume kube-api-access-k2l25 for pod openshift-network-diagnostics/network-check-target-ftxlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:18.483679 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.483597 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25 podName:a6cf0904-b4aa-403b-b630-4281b5bcd835 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:19.483564372 +0000 UTC m=+25.320269148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2l25" (UniqueName: "kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25") pod "network-check-target-ftxlr" (UID: "a6cf0904-b4aa-403b-b630-4281b5bcd835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:18.659340 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.659308 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 19:16:18.668279 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.668247 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/320343bc-1b3b-4ca7-9ae4-1e073451be95-cni-binary-copy\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:18.668488 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.668247 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-cni-binary-copy\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:18.680787 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.680759 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 19:16:18.740682 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.740609 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:18.740682 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.740654 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:18.740888 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.740776 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:18.740888 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.740869 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:18.778543 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.778512 2562 configmap.go:193] Couldn't get configMap openshift-multus/multus-daemon-config: failed to sync configmap cache: timed out waiting for the condition Jul 15 19:16:18.778705 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.778622 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-daemon-config podName:7b0824c5-d7a8-4c30-91f5-f2b60f034e4d nodeName:}" failed. No retries permitted until 2026-07-15 19:16:19.278600018 +0000 UTC m=+25.115304779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "multus-daemon-config" (UniqueName: "kubernetes.io/configmap/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-daemon-config") pod "multus-btwt9" (UID: "7b0824c5-d7a8-4c30-91f5-f2b60f034e4d") : failed to sync configmap cache: timed out waiting for the condition Jul 15 19:16:18.802724 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.802690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxc2j" event={"ID":"a87c2704-0fe6-4050-87d0-cc80a8da963b","Type":"ContainerStarted","Data":"29860609b68d766daf50d64ba2f38712f9ff0a5e890073821f46bd0b7e3bfe1e"} Jul 15 19:16:18.804983 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.804953 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerStarted","Data":"c617f56d2685e4c580856ee32639809875481c199b647043afa502c44bb9c997"} Jul 15 19:16:18.807720 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.807695 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z6fkb" event={"ID":"aa06f283-9e57-4543-a309-5605c58b0893","Type":"ContainerStarted","Data":"1554e62d60ad2f92855483656ca9e39ec25574c925a2b34d13b9e981cf521f7b"} Jul 15 19:16:18.810522 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.809857 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal_b50a42c21712f0bb6adb1e8ff74aa182/setup/0.log" Jul 15 19:16:18.810522 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.809890 2562 generic.go:356] "Generic (PLEG): container finished" podID="b50a42c21712f0bb6adb1e8ff74aa182" containerID="fd6cb4bbfe545582c647d6559a7b1d70e0775f6c2cec99a9238f307e19da1287" exitCode=0 Jul 15 19:16:18.810522 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.809926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" event={"ID":"b50a42c21712f0bb6adb1e8ff74aa182","Type":"ContainerDied","Data":"fd6cb4bbfe545582c647d6559a7b1d70e0775f6c2cec99a9238f307e19da1287"} Jul 15 19:16:18.810522 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.809966 2562 scope.go:117] "RemoveContainer" containerID="7df1c4e34b42c0cab7e17bf3729123197c4e0b13f0256e11733637817719cf10" Jul 15 19:16:18.810522 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.810351 2562 scope.go:117] "RemoveContainer" containerID="7df1c4e34b42c0cab7e17bf3729123197c4e0b13f0256e11733637817719cf10" Jul 15 19:16:18.825105 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.825072 2562 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal_openshift-machine-config-operator_b50a42c21712f0bb6adb1e8ff74aa182_0 in pod sandbox 1a16cbfceb86156cf1643a3a4aacf3ae556716ca703b2b8d6831ed06ef201dae from index: no such id: '7df1c4e34b42c0cab7e17bf3729123197c4e0b13f0256e11733637817719cf10'" containerID="7df1c4e34b42c0cab7e17bf3729123197c4e0b13f0256e11733637817719cf10" Jul 15 19:16:18.825213 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:18.825129 2562 kuberuntime_container.go:962] "Unhandled Error" err="failed to remove pod init container \"setup\": rpc error: code = Unknown desc = failed to delete container k8s_setup_kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal_openshift-machine-config-operator_b50a42c21712f0bb6adb1e8ff74aa182_0 in pod sandbox 1a16cbfceb86156cf1643a3a4aacf3ae556716ca703b2b8d6831ed06ef201dae from index: no such id: '7df1c4e34b42c0cab7e17bf3729123197c4e0b13f0256e11733637817719cf10'; Skipping pod \"kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal_openshift-machine-config-operator(b50a42c21712f0bb6adb1e8ff74aa182)\"" logger="UnhandledError" Jul 15 19:16:18.973961 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:18.973927 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ncmhl\"" Jul 15 19:16:19.005418 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.005354 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 19:16:19.017985 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.017954 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq4bj\" (UniqueName: \"kubernetes.io/projected/320343bc-1b3b-4ca7-9ae4-1e073451be95-kube-api-access-fq4bj\") pod \"multus-additional-cni-plugins-cdt4d\" (UID: \"320343bc-1b3b-4ca7-9ae4-1e073451be95\") " pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:19.018128 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.017959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcxjr\" (UniqueName: \"kubernetes.io/projected/b087f738-59f9-4bcb-b227-33cfdee6adc9-kube-api-access-jcxjr\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:19.027232 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.027200 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x2cd\" (UniqueName: \"kubernetes.io/projected/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-kube-api-access-5x2cd\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:19.065461 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.065433 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 19:16:19.109091 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.109055 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" Jul 15 19:16:19.122591 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:16:19.122536 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320343bc_1b3b_4ca7_9ae4_1e073451be95.slice/crio-cb19b592855dffd183b11668a2f9a02bf306e0f62c46d954e2c1886ae8291da5 WatchSource:0}: Error finding container cb19b592855dffd183b11668a2f9a02bf306e0f62c46d954e2c1886ae8291da5: Status 404 returned error can't find the container with id cb19b592855dffd183b11668a2f9a02bf306e0f62c46d954e2c1886ae8291da5 Jul 15 19:16:19.188719 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.188683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:19.188929 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:19.188879 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:19.189001 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:19.188943 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret podName:648cd87d-755f-4648-97d3-0857e63a32bb nodeName:}" failed. No retries permitted until 2026-07-15 19:16:21.188925667 +0000 UTC m=+27.025630427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret") pod "global-pull-secret-syncer-j8jpj" (UID: "648cd87d-755f-4648-97d3-0857e63a32bb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:19.290114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.290035 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:19.290255 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.290126 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-daemon-config\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:19.290255 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:19.290174 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:19.290361 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:19.290258 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs podName:b087f738-59f9-4bcb-b227-33cfdee6adc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:21.290233529 +0000 UTC m=+27.126938304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs") pod "network-metrics-daemon-gtxhr" (UID: "b087f738-59f9-4bcb-b227-33cfdee6adc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:19.290748 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.290723 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b0824c5-d7a8-4c30-91f5-f2b60f034e4d-multus-daemon-config\") pod \"multus-btwt9\" (UID: \"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d\") " pod="openshift-multus/multus-btwt9" Jul 15 19:16:19.401415 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.401377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-btwt9" Jul 15 19:16:19.444063 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.443998 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:11:17 +0000 UTC" deadline="2028-01-28 22:08:59.575311064 +0000 UTC" Jul 15 19:16:19.444063 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.444061 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13490h52m40.131254358s" Jul 15 19:16:19.491277 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.491177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:19.491451 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:19.491371 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:16:19.491451 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:19.491395 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:16:19.491451 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:19.491409 2562 projected.go:196] Error preparing data for projected volume kube-api-access-k2l25 for pod openshift-network-diagnostics/network-check-target-ftxlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:19.491619 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:19.491475 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25 podName:a6cf0904-b4aa-403b-b630-4281b5bcd835 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:21.491454342 +0000 UTC m=+27.328159104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2l25" (UniqueName: "kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25") pod "network-check-target-ftxlr" (UID: "a6cf0904-b4aa-403b-b630-4281b5bcd835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:19.737926 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.737828 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:19.738079 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:19.737957 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:19.814274 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.814230 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" event={"ID":"b50a42c21712f0bb6adb1e8ff74aa182","Type":"ContainerStarted","Data":"530d3c665936810f3e2508b91712e5c43e998bec67f6f749e145c56ed5aff14f"} Jul 15 19:16:19.815489 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:19.815461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" event={"ID":"320343bc-1b3b-4ca7-9ae4-1e073451be95","Type":"ContainerStarted","Data":"cb19b592855dffd183b11668a2f9a02bf306e0f62c46d954e2c1886ae8291da5"} Jul 15 19:16:20.741016 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:20.740989 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:20.741466 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:20.741109 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:20.741466 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:20.741158 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:20.741466 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:20.741241 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:21.206508 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.206422 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:21.206678 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:21.206556 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:21.206678 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:21.206630 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret podName:648cd87d-755f-4648-97d3-0857e63a32bb nodeName:}" failed. No retries permitted until 2026-07-15 19:16:25.206614536 +0000 UTC m=+31.043319304 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret") pod "global-pull-secret-syncer-j8jpj" (UID: "648cd87d-755f-4648-97d3-0857e63a32bb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:21.243123 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:16:21.243091 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b0824c5_d7a8_4c30_91f5_f2b60f034e4d.slice/crio-b2c8658afb8729f694206939dc6b484dd867a54bab988a49c24e2a66b532237c WatchSource:0}: Error finding container b2c8658afb8729f694206939dc6b484dd867a54bab988a49c24e2a66b532237c: Status 404 returned error can't find the container with id b2c8658afb8729f694206939dc6b484dd867a54bab988a49c24e2a66b532237c Jul 15 19:16:21.307692 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.307661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:21.307854 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:21.307814 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:21.307919 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:21.307887 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs podName:b087f738-59f9-4bcb-b227-33cfdee6adc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:25.307866331 +0000 UTC m=+31.144571105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs") pod "network-metrics-daemon-gtxhr" (UID: "b087f738-59f9-4bcb-b227-33cfdee6adc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:21.510201 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.510107 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:21.510331 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:21.510251 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:16:21.510331 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:21.510276 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:16:21.510331 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:21.510290 2562 projected.go:196] Error preparing data for projected volume kube-api-access-k2l25 for pod openshift-network-diagnostics/network-check-target-ftxlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:21.510441 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:21.510352 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25 podName:a6cf0904-b4aa-403b-b630-4281b5bcd835 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:25.510334073 +0000 UTC m=+31.347038834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2l25" (UniqueName: "kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25") pod "network-check-target-ftxlr" (UID: "a6cf0904-b4aa-403b-b630-4281b5bcd835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:21.737762 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.737726 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:21.737912 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:21.737857 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:21.821921 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.821831 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-btwt9" event={"ID":"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d","Type":"ContainerStarted","Data":"b2c8658afb8729f694206939dc6b484dd867a54bab988a49c24e2a66b532237c"} Jul 15 19:16:21.823975 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.823946 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxc2j" event={"ID":"a87c2704-0fe6-4050-87d0-cc80a8da963b","Type":"ContainerStarted","Data":"0db04a99b0d85ce239cb45e25582f3a4c8a37cc4154b797a7c5310e500a7de92"} Jul 15 19:16:21.825855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.825827 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-z6fkb" event={"ID":"aa06f283-9e57-4543-a309-5605c58b0893","Type":"ContainerStarted","Data":"482446c9125502ab0c5875018ce4e3cc450cc5042096326fc4a0445adeb2a411"} Jul 15 19:16:21.838969 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.838919 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-236.ec2.internal" podStartSLOduration=26.8389045 podStartE2EDuration="26.8389045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:15:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:19.83103336 +0000 UTC m=+25.667738137" watchObservedRunningTime="2026-07-15 19:16:21.8389045 +0000 UTC m=+27.675609290" Jul 15 19:16:21.839119 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.839026 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vxc2j" podStartSLOduration=1.420999484 podStartE2EDuration="4.839020295s" totalImagesPullingTime="3.418020811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:21.837814187 +0000 UTC m=+27.674518968" watchObservedRunningTime="2026-07-15 19:16:21.839020295 +0000 UTC m=+27.675725072" Jul 15 19:16:21.851113 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:21.851074 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-z6fkb" podStartSLOduration=1.478937188 podStartE2EDuration="4.851060942s" totalImagesPullingTime="3.372123754s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:21.850653249 +0000 UTC m=+27.687358027" watchObservedRunningTime="2026-07-15 19:16:21.851060942 +0000 UTC m=+27.687765718" Jul 15 19:16:22.740872 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:22.740795 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:22.741031 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:22.740932 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:22.741305 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:22.741286 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:22.741423 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:22.741384 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:22.814595 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:22.814503 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:22.815160 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:22.815139 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:22.829659 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:22.829628 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:22.835105 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:22.833728 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vxc2j" Jul 15 19:16:23.736937 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:23.736897 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:23.737113 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:23.737030 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:24.738755 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:24.738171 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:24.738755 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:24.738266 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:24.738755 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:24.738623 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:24.738755 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:24.738712 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:24.835967 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:24.835930 2562 generic.go:356] "Generic (PLEG): container finished" podID="320343bc-1b3b-4ca7-9ae4-1e073451be95" containerID="2ba82612692e0e6b3a9085f5435462791974ad4758ba46069e17e61c733c5e2b" exitCode=0 Jul 15 19:16:24.836799 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:24.836770 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" event={"ID":"320343bc-1b3b-4ca7-9ae4-1e073451be95","Type":"ContainerDied","Data":"2ba82612692e0e6b3a9085f5435462791974ad4758ba46069e17e61c733c5e2b"} Jul 15 19:16:25.243960 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:25.243924 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:25.244137 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:25.244086 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:25.244201 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:25.244158 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret podName:648cd87d-755f-4648-97d3-0857e63a32bb nodeName:}" failed. No retries permitted until 2026-07-15 19:16:33.244139794 +0000 UTC m=+39.080844549 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret") pod "global-pull-secret-syncer-j8jpj" (UID: "648cd87d-755f-4648-97d3-0857e63a32bb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:25.344374 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:25.344339 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:25.344543 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:25.344510 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:25.344611 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:25.344596 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs podName:b087f738-59f9-4bcb-b227-33cfdee6adc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:33.344558224 +0000 UTC m=+39.181262983 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs") pod "network-metrics-daemon-gtxhr" (UID: "b087f738-59f9-4bcb-b227-33cfdee6adc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:25.546443 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:25.546349 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:25.546670 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:25.546540 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:16:25.546670 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:25.546561 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:16:25.546670 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:25.546590 2562 projected.go:196] Error preparing data for projected volume kube-api-access-k2l25 for pod openshift-network-diagnostics/network-check-target-ftxlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:25.546670 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:25.546654 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25 podName:a6cf0904-b4aa-403b-b630-4281b5bcd835 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:33.54663428 +0000 UTC m=+39.383339049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2l25" (UniqueName: "kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25") pod "network-check-target-ftxlr" (UID: "a6cf0904-b4aa-403b-b630-4281b5bcd835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:25.737732 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:25.737696 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:25.737891 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:25.737822 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:26.737046 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:26.737013 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:26.737501 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:26.737152 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:26.737501 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:26.737206 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:26.737501 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:26.737332 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:27.736911 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:27.736882 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:27.737095 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:27.737007 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:28.737112 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:28.737074 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:28.737510 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:28.737217 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:28.737510 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:28.737255 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:28.737510 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:28.737338 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:29.737938 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:29.737900 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:29.738389 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:29.738036 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:30.541755 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:30.541719 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z6fkb_aa06f283-9e57-4543-a309-5605c58b0893/node-ca/0.log" Jul 15 19:16:30.738899 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:30.738723 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:30.739269 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:30.738793 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:30.739319 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:30.739243 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:30.741847 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:30.739890 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:31.737473 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:31.737434 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:31.737671 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:31.737564 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:32.737393 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:32.737297 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:32.737393 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:32.737298 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:32.737881 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:32.737441 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:32.737881 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:32.737475 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:33.304747 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:33.304707 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:33.304938 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:33.304828 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:33.304938 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:33.304886 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret podName:648cd87d-755f-4648-97d3-0857e63a32bb nodeName:}" failed. No retries permitted until 2026-07-15 19:16:49.304871425 +0000 UTC m=+55.141576196 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret") pod "global-pull-secret-syncer-j8jpj" (UID: "648cd87d-755f-4648-97d3-0857e63a32bb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:33.405283 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:33.405248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:33.405455 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:33.405397 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:33.405455 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:33.405455 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs podName:b087f738-59f9-4bcb-b227-33cfdee6adc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:49.405440284 +0000 UTC m=+55.242145040 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs") pod "network-metrics-daemon-gtxhr" (UID: "b087f738-59f9-4bcb-b227-33cfdee6adc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:33.606968 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:33.606892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:33.607179 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:33.607052 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:16:33.607179 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:33.607080 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:16:33.607179 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:33.607094 2562 projected.go:196] Error preparing data for projected volume kube-api-access-k2l25 for pod openshift-network-diagnostics/network-check-target-ftxlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:33.607179 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:33.607152 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25 podName:a6cf0904-b4aa-403b-b630-4281b5bcd835 nodeName:}" failed. No retries permitted until 2026-07-15 19:16:49.607135505 +0000 UTC m=+55.443840299 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2l25" (UniqueName: "kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25") pod "network-check-target-ftxlr" (UID: "a6cf0904-b4aa-403b-b630-4281b5bcd835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:33.737859 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:33.737821 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:33.738405 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:33.737948 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:34.740202 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:34.738775 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:34.740202 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:34.738920 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:34.740202 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:34.739391 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:34.740202 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:34.739534 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:35.736917 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:35.736885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:35.737061 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:35.736990 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:36.741105 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.740825 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:36.741779 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.740883 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:36.741779 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:36.741193 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:36.741779 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:36.741263 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:36.863083 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.863059 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:16:36.863403 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.863383 2562 generic.go:356] "Generic (PLEG): container finished" podID="b1d40045-d2ca-4b74-8192-248a35efdf7d" containerID="f9ac267ef29de2b00304bdd462a844b650739f71f2f70dc7488ddbb0298d82c2" exitCode=1 Jul 15 19:16:36.863485 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.863459 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerStarted","Data":"deaedafcc19c2a07079baf330c6eb071deca14c67e55edb19265b2e5dcdc000c"} Jul 15 19:16:36.863556 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.863543 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerStarted","Data":"0fde1b861bca000f32220bf838e4989f360c2cd465d9ce1dc2bb180452ab7db6"} Jul 15 19:16:36.863633 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.863613 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerStarted","Data":"e23d8eb3a854b7427492f3127f8942168d56794c9a749dad869825b19d547ff1"} Jul 15 19:16:36.863686 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.863664 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerStarted","Data":"712264f179ef02cc00321da26771439454bf0e0b2a6118c3b2f68fdf6c095c42"} Jul 15 19:16:36.863740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.863710 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerDied","Data":"f9ac267ef29de2b00304bdd462a844b650739f71f2f70dc7488ddbb0298d82c2"} Jul 15 19:16:36.863791 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.863763 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerStarted","Data":"c79710f1a1116a8fe78876b8706fd6e2f2467e3e2025869a8c68d3a5fa649287"} Jul 15 19:16:36.865055 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.865037 2562 generic.go:356] "Generic (PLEG): container finished" podID="320343bc-1b3b-4ca7-9ae4-1e073451be95" containerID="301bdb728ea8e3f2e1c7a7de224f9b21b684e517041877fced7f02a6e656072b" exitCode=0 Jul 15 19:16:36.865145 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.865099 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" event={"ID":"320343bc-1b3b-4ca7-9ae4-1e073451be95","Type":"ContainerDied","Data":"301bdb728ea8e3f2e1c7a7de224f9b21b684e517041877fced7f02a6e656072b"} Jul 15 19:16:36.869688 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.869662 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-btwt9" event={"ID":"7b0824c5-d7a8-4c30-91f5-f2b60f034e4d","Type":"ContainerStarted","Data":"7bafe3c14fdc4140567f618a9cc8314edfae237ec64d8885fabd5fb4b3767b26"} Jul 15 19:16:36.926734 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:36.926599 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-btwt9" podStartSLOduration=4.999286784 podStartE2EDuration="19.92656478s" totalImagesPullingTime="14.927277996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:36.926419649 +0000 UTC m=+42.763124426" watchObservedRunningTime="2026-07-15 19:16:36.92656478 +0000 UTC m=+42.763269555" Jul 15 19:16:37.737702 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:37.737671 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:37.737896 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:37.737761 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:37.873855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:37.873770 2562 generic.go:356] "Generic (PLEG): container finished" podID="320343bc-1b3b-4ca7-9ae4-1e073451be95" containerID="a7dc719a2c3f695aad5b7a8216a7802c2d00303ed65dacbc884638ad66c56066" exitCode=0 Jul 15 19:16:37.874209 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:37.873854 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" event={"ID":"320343bc-1b3b-4ca7-9ae4-1e073451be95","Type":"ContainerDied","Data":"a7dc719a2c3f695aad5b7a8216a7802c2d00303ed65dacbc884638ad66c56066"} Jul 15 19:16:38.739750 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:38.739672 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:38.739883 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:38.739773 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:38.739883 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:38.739799 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:38.740222 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:38.740193 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:38.879336 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:38.879314 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:16:38.882401 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:38.882373 2562 generic.go:356] "Generic (PLEG): container finished" podID="320343bc-1b3b-4ca7-9ae4-1e073451be95" containerID="32b39454f4346c69030840c0f6b0bc3cf6d2d7f4ba0d72e580656d4fb3ff18fb" exitCode=0 Jul 15 19:16:38.882533 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:38.882429 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" event={"ID":"320343bc-1b3b-4ca7-9ae4-1e073451be95","Type":"ContainerDied","Data":"32b39454f4346c69030840c0f6b0bc3cf6d2d7f4ba0d72e580656d4fb3ff18fb"} Jul 15 19:16:39.737682 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:39.737652 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:39.737872 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:39.737759 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:39.889107 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:39.889076 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:16:39.889544 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:39.889431 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerStarted","Data":"950c88af5a5bef42d418d8f8cefa9ee4c7160558397fffcc765b298afd5ab8d1"} Jul 15 19:16:40.320908 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.320874 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jjbqn"] Jul 15 19:16:40.336915 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.336885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.340188 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.339731 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 19:16:40.340188 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.339757 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 19:16:40.340188 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.339734 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q2bf5\"" Jul 15 19:16:40.340188 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.340178 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 19:16:40.340913 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.340890 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 19:16:40.340913 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.340906 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 19:16:40.341285 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.341151 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 19:16:40.457793 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.457734 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-sys\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.457793 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.457784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-tls\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.458059 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.457810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.458059 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.457857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-wtmp\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.458059 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.457891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnwxm\" (UniqueName: \"kubernetes.io/projected/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-kube-api-access-tnwxm\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.458059 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.457936 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.458059 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.457962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-textfile\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.458059 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.457987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-metrics-client-ca\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.458059 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.458025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-root\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558402 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558372 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-sys\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558608 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-tls\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558608 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558608 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-wtmp\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558608 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnwxm\" (UniqueName: \"kubernetes.io/projected/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-kube-api-access-tnwxm\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558608 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558500 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-sys\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558608 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558608 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-textfile\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558608 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-metrics-client-ca\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558972 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558644 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-root\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.558972 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558771 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-root\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.559059 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.558975 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-wtmp\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.559162 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.559135 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.559525 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.559502 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-textfile\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.559627 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.559595 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-metrics-client-ca\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.562930 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.562908 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.563028 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.562915 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-node-exporter-tls\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.570236 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.570206 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnwxm\" (UniqueName: \"kubernetes.io/projected/2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0-kube-api-access-tnwxm\") pod \"node-exporter-jjbqn\" (UID: \"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0\") " pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.647295 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.647259 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jjbqn" Jul 15 19:16:40.683967 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:16:40.683941 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eb8ebd9_ff60_4e9e_aaa1_fe19738e61e0.slice/crio-403f0be06f43a80df302e260e0f67e6deea13694281d8e36d8fd8fae37b8b1db WatchSource:0}: Error finding container 403f0be06f43a80df302e260e0f67e6deea13694281d8e36d8fd8fae37b8b1db: Status 404 returned error can't find the container with id 403f0be06f43a80df302e260e0f67e6deea13694281d8e36d8fd8fae37b8b1db Jul 15 19:16:40.737613 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.737568 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:40.737769 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:40.737740 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:40.738227 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.738203 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:40.738347 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:40.738328 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:40.893173 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:40.893130 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jjbqn" event={"ID":"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0","Type":"ContainerStarted","Data":"403f0be06f43a80df302e260e0f67e6deea13694281d8e36d8fd8fae37b8b1db"} Jul 15 19:16:41.737518 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:41.737484 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:41.737724 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:41.737627 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:41.899450 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:41.899424 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:16:41.899992 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:41.899861 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerStarted","Data":"9b863555f63d4ee46b27024f8409976793e3824715b898b16407155cd113871f"} Jul 15 19:16:41.900216 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:41.900179 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:41.900345 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:41.900268 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:41.900345 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:41.900335 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:41.900440 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:41.900364 2562 scope.go:117] "RemoveContainer" containerID="f9ac267ef29de2b00304bdd462a844b650739f71f2f70dc7488ddbb0298d82c2" Jul 15 19:16:41.918155 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:41.918128 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:41.918297 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:41.918254 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:16:42.737773 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:42.737517 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:42.737942 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:42.737607 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:42.737942 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:42.737883 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:42.738057 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:42.737970 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:43.317503 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.317468 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j8jpj"] Jul 15 19:16:43.317979 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.317645 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:43.317979 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:43.317751 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:43.320767 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.320724 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ftxlr"] Jul 15 19:16:43.320895 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.320842 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:43.320967 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:43.320948 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:43.321474 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.321453 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gtxhr"] Jul 15 19:16:43.321614 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.321539 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:43.321692 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:43.321668 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:43.908782 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.908756 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:16:43.909139 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.909114 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" event={"ID":"b1d40045-d2ca-4b74-8192-248a35efdf7d","Type":"ContainerStarted","Data":"ab15fa1849e2afa4395866f570488d0358da7cee773bd68e2514d4933deca5ea"} Jul 15 19:16:43.913938 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.913914 2562 generic.go:356] "Generic (PLEG): container finished" podID="320343bc-1b3b-4ca7-9ae4-1e073451be95" containerID="cb07102f313edfe2832b202526ee2957574d9104279242480241c78788b85043" exitCode=0 Jul 15 19:16:43.914052 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.913980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" event={"ID":"320343bc-1b3b-4ca7-9ae4-1e073451be95","Type":"ContainerDied","Data":"cb07102f313edfe2832b202526ee2957574d9104279242480241c78788b85043"} Jul 15 19:16:43.915607 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.915559 2562 generic.go:356] "Generic (PLEG): container finished" podID="2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0" containerID="f10d87f6e575c80900194c762ad73e0b7e0150b00119d47cf98aa7730fdd39ef" exitCode=0 Jul 15 19:16:43.915706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.915604 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jjbqn" event={"ID":"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0","Type":"ContainerDied","Data":"f10d87f6e575c80900194c762ad73e0b7e0150b00119d47cf98aa7730fdd39ef"} Jul 15 19:16:43.942606 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:43.942525 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" podStartSLOduration=8.803448585 podStartE2EDuration="26.942503279s" totalImagesPullingTime="18.139054694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:43.941601775 +0000 UTC m=+49.778306551" watchObservedRunningTime="2026-07-15 19:16:43.942503279 +0000 UTC m=+49.779208058" Jul 15 19:16:44.737867 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:44.737830 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:44.738591 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:44.737933 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:44.738591 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:44.738025 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:44.738591 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:44.738121 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:44.738591 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:44.738173 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:44.738591 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:44.738250 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:44.920829 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:44.920799 2562 generic.go:356] "Generic (PLEG): container finished" podID="320343bc-1b3b-4ca7-9ae4-1e073451be95" containerID="e0e012ef51b98d01bd95a4f49329047a8229253d1c5908df3e8529f01ea8007d" exitCode=0 Jul 15 19:16:44.920993 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:44.920837 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" event={"ID":"320343bc-1b3b-4ca7-9ae4-1e073451be95","Type":"ContainerDied","Data":"e0e012ef51b98d01bd95a4f49329047a8229253d1c5908df3e8529f01ea8007d"} Jul 15 19:16:44.923002 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:44.922974 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jjbqn" event={"ID":"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0","Type":"ContainerStarted","Data":"3fe15a14b1fb833f83a0a0fba60d6184a3f25e485c271de50971bfa1e57ab128"} Jul 15 19:16:44.923112 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:44.923031 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jjbqn" event={"ID":"2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0","Type":"ContainerStarted","Data":"bbfedd62e03de43c56fe6ed123fa278d72aa932c0649b5452c7d36052703df05"} Jul 15 19:16:45.931398 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:45.931204 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" event={"ID":"320343bc-1b3b-4ca7-9ae4-1e073451be95","Type":"ContainerStarted","Data":"7e8990dcc0179d49deb583a462d26abcb346bf8b328afa3aa6032d618acc1655"} Jul 15 19:16:45.957739 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:45.957686 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cdt4d" podStartSLOduration=6.8772776 podStartE2EDuration="28.957670626s" totalImagesPullingTime="22.080393026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:16:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:45.957271796 +0000 UTC m=+51.793976573" watchObservedRunningTime="2026-07-15 19:16:45.957670626 +0000 UTC m=+51.794375383" Jul 15 19:16:45.958133 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:45.958060 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jjbqn" podStartSLOduration=3.1695467170000002 podStartE2EDuration="5.958047914s" totalImagesPullingTime="2.673743284s" totalInitContainerRuntime="114.757913ms" isStatefulPod=true podCreationTimestamp="2026-07-15 19:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:44.96987345 +0000 UTC m=+50.806578227" watchObservedRunningTime="2026-07-15 19:16:45.958047914 +0000 UTC m=+51.794752674" Jul 15 19:16:46.737867 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:46.737832 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:46.738055 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:46.737930 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:46.738055 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:46.737837 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:46.738055 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:46.737976 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:46.738215 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:46.738053 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:46.738215 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:46.738121 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:48.737528 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:48.737488 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:48.737528 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:48.737528 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:48.737931 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:48.737620 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:48.737931 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:48.737643 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ftxlr" podUID="a6cf0904-b4aa-403b-b630-4281b5bcd835" Jul 15 19:16:48.737931 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:48.737701 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gtxhr" podUID="b087f738-59f9-4bcb-b227-33cfdee6adc9" Jul 15 19:16:48.737931 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:48.737754 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8jpj" podUID="648cd87d-755f-4648-97d3-0857e63a32bb" Jul 15 19:16:48.957648 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:48.957621 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-236.ec2.internal" event="NodeReady" Jul 15 19:16:48.957814 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:48.957743 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 19:16:49.001390 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.001362 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dpnpp"] Jul 15 19:16:49.008386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.008361 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.009399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.009376 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lzdk7"] Jul 15 19:16:49.011434 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.011406 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 19:16:49.011530 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.011439 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8qrjh\"" Jul 15 19:16:49.012034 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.012018 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 19:16:49.016645 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.016627 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dpnpp"] Jul 15 19:16:49.016739 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.016727 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.019243 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.019226 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 19:16:49.019428 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.019387 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 19:16:49.019528 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.019281 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 19:16:49.019604 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.019547 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 19:16:49.019713 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.019696 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bl2wc\"" Jul 15 19:16:49.024339 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.024318 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzdk7"] Jul 15 19:16:49.027518 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.027489 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m5dcq"] Jul 15 19:16:49.035103 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.035081 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m5dcq" Jul 15 19:16:49.038120 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.038098 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 19:16:49.038222 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.038179 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 19:16:49.038469 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.038440 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 19:16:49.038522 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.038478 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-swcsq\"" Jul 15 19:16:49.041347 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.041327 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m5dcq"] Jul 15 19:16:49.129514 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rh4b\" (UniqueName: \"kubernetes.io/projected/8ca0c146-c33d-45e7-9f93-4b004774d54a-kube-api-access-7rh4b\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.129514 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129514 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56161d04-16b3-4de2-8411-38c3c90b2de4-crio-socket\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.129777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ca0c146-c33d-45e7-9f93-4b004774d54a-metrics-tls\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.129777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129638 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56161d04-16b3-4de2-8411-38c3c90b2de4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.129777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56161d04-16b3-4de2-8411-38c3c90b2de4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.129777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d2pm\" (UniqueName: \"kubernetes.io/projected/56161d04-16b3-4de2-8411-38c3c90b2de4-kube-api-access-7d2pm\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.129777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56161d04-16b3-4de2-8411-38c3c90b2de4-data-volume\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.129777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129777 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca0c146-c33d-45e7-9f93-4b004774d54a-config-volume\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.130044 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129808 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ca0c146-c33d-45e7-9f93-4b004774d54a-tmp-dir\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.130044 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86e9f1ce-b8c5-4eed-a402-757a5d74501f-cert\") pod \"ingress-canary-m5dcq\" (UID: \"86e9f1ce-b8c5-4eed-a402-757a5d74501f\") " pod="openshift-ingress-canary/ingress-canary-m5dcq" Jul 15 19:16:49.130044 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.129883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qbxp\" (UniqueName: \"kubernetes.io/projected/86e9f1ce-b8c5-4eed-a402-757a5d74501f-kube-api-access-7qbxp\") pod \"ingress-canary-m5dcq\" (UID: \"86e9f1ce-b8c5-4eed-a402-757a5d74501f\") " pod="openshift-ingress-canary/ingress-canary-m5dcq" Jul 15 19:16:49.230881 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.230795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56161d04-16b3-4de2-8411-38c3c90b2de4-data-volume\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.230881 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.230833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca0c146-c33d-45e7-9f93-4b004774d54a-config-volume\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.230881 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.230862 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ca0c146-c33d-45e7-9f93-4b004774d54a-tmp-dir\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.231150 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.230905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86e9f1ce-b8c5-4eed-a402-757a5d74501f-cert\") pod \"ingress-canary-m5dcq\" (UID: \"86e9f1ce-b8c5-4eed-a402-757a5d74501f\") " pod="openshift-ingress-canary/ingress-canary-m5dcq" Jul 15 19:16:49.231150 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.230926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qbxp\" (UniqueName: \"kubernetes.io/projected/86e9f1ce-b8c5-4eed-a402-757a5d74501f-kube-api-access-7qbxp\") pod \"ingress-canary-m5dcq\" (UID: \"86e9f1ce-b8c5-4eed-a402-757a5d74501f\") " pod="openshift-ingress-canary/ingress-canary-m5dcq" Jul 15 19:16:49.231150 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.230985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rh4b\" (UniqueName: \"kubernetes.io/projected/8ca0c146-c33d-45e7-9f93-4b004774d54a-kube-api-access-7rh4b\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.231150 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56161d04-16b3-4de2-8411-38c3c90b2de4-crio-socket\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.231150 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ca0c146-c33d-45e7-9f93-4b004774d54a-metrics-tls\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.231400 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56161d04-16b3-4de2-8411-38c3c90b2de4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.231400 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231186 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/56161d04-16b3-4de2-8411-38c3c90b2de4-data-volume\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.231400 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56161d04-16b3-4de2-8411-38c3c90b2de4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.231400 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231236 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d2pm\" (UniqueName: \"kubernetes.io/projected/56161d04-16b3-4de2-8411-38c3c90b2de4-kube-api-access-7d2pm\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.231400 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231260 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ca0c146-c33d-45e7-9f93-4b004774d54a-tmp-dir\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.231400 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231310 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/56161d04-16b3-4de2-8411-38c3c90b2de4-crio-socket\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.231712 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231686 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8ca0c146-c33d-45e7-9f93-4b004774d54a-config-volume\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.231812 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.231793 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/56161d04-16b3-4de2-8411-38c3c90b2de4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.235046 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.235025 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/56161d04-16b3-4de2-8411-38c3c90b2de4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.235169 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.235066 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ca0c146-c33d-45e7-9f93-4b004774d54a-metrics-tls\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.235228 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.235170 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86e9f1ce-b8c5-4eed-a402-757a5d74501f-cert\") pod \"ingress-canary-m5dcq\" (UID: \"86e9f1ce-b8c5-4eed-a402-757a5d74501f\") " pod="openshift-ingress-canary/ingress-canary-m5dcq" Jul 15 19:16:49.240755 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.240728 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rh4b\" (UniqueName: \"kubernetes.io/projected/8ca0c146-c33d-45e7-9f93-4b004774d54a-kube-api-access-7rh4b\") pod \"dns-default-dpnpp\" (UID: \"8ca0c146-c33d-45e7-9f93-4b004774d54a\") " pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.241202 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.241182 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d2pm\" (UniqueName: \"kubernetes.io/projected/56161d04-16b3-4de2-8411-38c3c90b2de4-kube-api-access-7d2pm\") pod \"insights-runtime-extractor-lzdk7\" (UID: \"56161d04-16b3-4de2-8411-38c3c90b2de4\") " pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.241513 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.241493 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qbxp\" (UniqueName: \"kubernetes.io/projected/86e9f1ce-b8c5-4eed-a402-757a5d74501f-kube-api-access-7qbxp\") pod \"ingress-canary-m5dcq\" (UID: \"86e9f1ce-b8c5-4eed-a402-757a5d74501f\") " pod="openshift-ingress-canary/ingress-canary-m5dcq" Jul 15 19:16:49.318319 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.318281 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:49.325950 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.325925 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lzdk7" Jul 15 19:16:49.332110 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.332087 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:49.332264 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:49.332247 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:49.332334 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:49.332323 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret podName:648cd87d-755f-4648-97d3-0857e63a32bb nodeName:}" failed. No retries permitted until 2026-07-15 19:17:21.332300202 +0000 UTC m=+87.169004974 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret") pod "global-pull-secret-syncer-j8jpj" (UID: "648cd87d-755f-4648-97d3-0857e63a32bb") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:16:49.342999 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.342974 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m5dcq" Jul 15 19:16:49.434603 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.433894 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:49.434603 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:49.434121 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:49.434603 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:49.434183 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs podName:b087f738-59f9-4bcb-b227-33cfdee6adc9 nodeName:}" failed. No retries permitted until 2026-07-15 19:17:21.434165393 +0000 UTC m=+87.270870157 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs") pod "network-metrics-daemon-gtxhr" (UID: "b087f738-59f9-4bcb-b227-33cfdee6adc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:16:49.511387 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.511303 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lzdk7"] Jul 15 19:16:49.516736 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.516708 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dpnpp"] Jul 15 19:16:49.517091 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:16:49.517068 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56161d04_16b3_4de2_8411_38c3c90b2de4.slice/crio-c633399c2ffa3e08b1f9af9456c50c2a5f1d7b6b893df12b481533bf2bcea04c WatchSource:0}: Error finding container c633399c2ffa3e08b1f9af9456c50c2a5f1d7b6b893df12b481533bf2bcea04c: Status 404 returned error can't find the container with id c633399c2ffa3e08b1f9af9456c50c2a5f1d7b6b893df12b481533bf2bcea04c Jul 15 19:16:49.519728 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:16:49.519707 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca0c146_c33d_45e7_9f93_4b004774d54a.slice/crio-54030ed2d5de34dc3112847e206e3646bba96cc6f0da038211cdd3334a21dc53 WatchSource:0}: Error finding container 54030ed2d5de34dc3112847e206e3646bba96cc6f0da038211cdd3334a21dc53: Status 404 returned error can't find the container with id 54030ed2d5de34dc3112847e206e3646bba96cc6f0da038211cdd3334a21dc53 Jul 15 19:16:49.523152 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.523132 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m5dcq"] Jul 15 19:16:49.525916 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:16:49.525887 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86e9f1ce_b8c5_4eed_a402_757a5d74501f.slice/crio-71c6306b9a9a67133d8a8757f7d8f4cb0eaa972df6ce5449eb33bcd61a8a070d WatchSource:0}: Error finding container 71c6306b9a9a67133d8a8757f7d8f4cb0eaa972df6ce5449eb33bcd61a8a070d: Status 404 returned error can't find the container with id 71c6306b9a9a67133d8a8757f7d8f4cb0eaa972df6ce5449eb33bcd61a8a070d Jul 15 19:16:49.636022 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.635983 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:49.636196 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:49.636109 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:16:49.636196 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:49.636133 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:16:49.636196 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:49.636142 2562 projected.go:196] Error preparing data for projected volume kube-api-access-k2l25 for pod openshift-network-diagnostics/network-check-target-ftxlr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:49.636196 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:16:49.636197 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25 podName:a6cf0904-b4aa-403b-b630-4281b5bcd835 nodeName:}" failed. No retries permitted until 2026-07-15 19:17:21.636180924 +0000 UTC m=+87.472885680 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k2l25" (UniqueName: "kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25") pod "network-check-target-ftxlr" (UID: "a6cf0904-b4aa-403b-b630-4281b5bcd835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:16:49.944694 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.944662 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m5dcq" event={"ID":"86e9f1ce-b8c5-4eed-a402-757a5d74501f","Type":"ContainerStarted","Data":"71c6306b9a9a67133d8a8757f7d8f4cb0eaa972df6ce5449eb33bcd61a8a070d"} Jul 15 19:16:49.945781 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.945757 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dpnpp" event={"ID":"8ca0c146-c33d-45e7-9f93-4b004774d54a","Type":"ContainerStarted","Data":"54030ed2d5de34dc3112847e206e3646bba96cc6f0da038211cdd3334a21dc53"} Jul 15 19:16:49.946786 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:49.946764 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzdk7" event={"ID":"56161d04-16b3-4de2-8411-38c3c90b2de4","Type":"ContainerStarted","Data":"c633399c2ffa3e08b1f9af9456c50c2a5f1d7b6b893df12b481533bf2bcea04c"} Jul 15 19:16:50.738101 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:50.738065 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:16:50.738815 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:50.738768 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:16:50.739671 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:50.739156 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:16:50.741377 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:50.741142 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:16:50.742744 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:50.742720 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:16:50.742974 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:50.742957 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 19:16:50.743120 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:50.743100 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 19:16:50.743190 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:50.743176 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w6x7t\"" Jul 15 19:16:50.743466 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:50.743448 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pzvzj\"" Jul 15 19:16:56.969950 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:56.969833 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m5dcq" event={"ID":"86e9f1ce-b8c5-4eed-a402-757a5d74501f","Type":"ContainerStarted","Data":"41ff43df5a7458a6a9d0daea0790116bbbbbbb91e7444ad5da6a691241f96c5b"} Jul 15 19:16:56.972020 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:56.971988 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dpnpp" event={"ID":"8ca0c146-c33d-45e7-9f93-4b004774d54a","Type":"ContainerStarted","Data":"d0d17b34fd36abd0562aae4fe5a664c73137a467ce82aaeae884eb3d9571b377"} Jul 15 19:16:56.973789 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:56.973765 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzdk7" event={"ID":"56161d04-16b3-4de2-8411-38c3c90b2de4","Type":"ContainerStarted","Data":"54eb31c3bed3694ea9f1c6b54618744e994039b232c274e3d28f1728b6169360"} Jul 15 19:16:56.987274 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:56.987215 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m5dcq" podStartSLOduration=1.684083463 podStartE2EDuration="7.987199965s" totalImagesPullingTime="6.303116502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:56.985643873 +0000 UTC m=+62.822348651" watchObservedRunningTime="2026-07-15 19:16:56.987199965 +0000 UTC m=+62.823904741" Jul 15 19:16:57.978957 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:57.978916 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dpnpp" event={"ID":"8ca0c146-c33d-45e7-9f93-4b004774d54a","Type":"ContainerStarted","Data":"02dde469e34106cb944ab12c59efe107391b79c892db23d98a4c624852499c2b"} Jul 15 19:16:57.979463 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:57.979139 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dpnpp" Jul 15 19:16:57.997908 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:16:57.997844 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dpnpp" podStartSLOduration=2.773614841 podStartE2EDuration="9.99782512s" totalImagesPullingTime="7.224210279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:16:57.99675324 +0000 UTC m=+63.833458016" watchObservedRunningTime="2026-07-15 19:16:57.99782512 +0000 UTC m=+63.834529876" Jul 15 19:17:00.992218 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:00.992170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzdk7" event={"ID":"56161d04-16b3-4de2-8411-38c3c90b2de4","Type":"ContainerStarted","Data":"62891a7dc12c07aa42afeda3bf357bff4b4e9c6c7ab125b96604efb690c2344f"} Jul 15 19:17:04.002510 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:04.002472 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lzdk7" event={"ID":"56161d04-16b3-4de2-8411-38c3c90b2de4","Type":"ContainerStarted","Data":"9bc14a96b5cf65dc0ad81e20183694f4884f876496e7c40cf2fe10b0fc80e5da"} Jul 15 19:17:04.020236 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:04.020184 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lzdk7" podStartSLOduration=2.095533956 podStartE2EDuration="16.02016911s" totalImagesPullingTime="13.924635154s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:16:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:04.019800683 +0000 UTC m=+69.856505461" watchObservedRunningTime="2026-07-15 19:17:04.02016911 +0000 UTC m=+69.856873887" Jul 15 19:17:07.984813 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:07.984785 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dpnpp" Jul 15 19:17:14.943532 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:14.943430 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9gs6l" Jul 15 19:17:21.383866 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.383827 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:17:21.386855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.386836 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:17:21.397043 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.397016 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/648cd87d-755f-4648-97d3-0857e63a32bb-original-pull-secret\") pod \"global-pull-secret-syncer-j8jpj\" (UID: \"648cd87d-755f-4648-97d3-0857e63a32bb\") " pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:17:21.485021 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.484981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:17:21.487639 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.487620 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:17:21.497981 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.497960 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b087f738-59f9-4bcb-b227-33cfdee6adc9-metrics-certs\") pod \"network-metrics-daemon-gtxhr\" (UID: \"b087f738-59f9-4bcb-b227-33cfdee6adc9\") " pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:17:21.655898 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.655803 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8jpj" Jul 15 19:17:21.667141 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.667102 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w6x7t\"" Jul 15 19:17:21.673645 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.673620 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gtxhr" Jul 15 19:17:21.686436 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.686405 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:17:21.689190 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.689169 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 19:17:21.700117 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.700088 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 19:17:21.711605 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.711560 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2l25\" (UniqueName: \"kubernetes.io/projected/a6cf0904-b4aa-403b-b630-4281b5bcd835-kube-api-access-k2l25\") pod \"network-check-target-ftxlr\" (UID: \"a6cf0904-b4aa-403b-b630-4281b5bcd835\") " pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:17:21.785022 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.784924 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j8jpj"] Jul 15 19:17:21.789067 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:17:21.789040 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648cd87d_755f_4648_97d3_0857e63a32bb.slice/crio-4292c9107cc81e2adc2fa459568e7af7929089a0ce537216a7574067e38591ef WatchSource:0}: Error finding container 4292c9107cc81e2adc2fa459568e7af7929089a0ce537216a7574067e38591ef: Status 404 returned error can't find the container with id 4292c9107cc81e2adc2fa459568e7af7929089a0ce537216a7574067e38591ef Jul 15 19:17:21.804852 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.804828 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gtxhr"] Jul 15 19:17:21.807043 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:17:21.807015 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb087f738_59f9_4bcb_b227_33cfdee6adc9.slice/crio-c799788d031341e20c702b7455a60a1e509497eb65ea320a382feee2e82c4aa2 WatchSource:0}: Error finding container c799788d031341e20c702b7455a60a1e509497eb65ea320a382feee2e82c4aa2: Status 404 returned error can't find the container with id c799788d031341e20c702b7455a60a1e509497eb65ea320a382feee2e82c4aa2 Jul 15 19:17:21.974237 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.974156 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pzvzj\"" Jul 15 19:17:21.981610 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:21.981569 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:17:22.055529 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:22.055500 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gtxhr" event={"ID":"b087f738-59f9-4bcb-b227-33cfdee6adc9","Type":"ContainerStarted","Data":"c799788d031341e20c702b7455a60a1e509497eb65ea320a382feee2e82c4aa2"} Jul 15 19:17:22.056636 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:22.056608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j8jpj" event={"ID":"648cd87d-755f-4648-97d3-0857e63a32bb","Type":"ContainerStarted","Data":"4292c9107cc81e2adc2fa459568e7af7929089a0ce537216a7574067e38591ef"} Jul 15 19:17:22.106217 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:22.106181 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ftxlr"] Jul 15 19:17:22.110181 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:17:22.110151 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6cf0904_b4aa_403b_b630_4281b5bcd835.slice/crio-ed2167ed0534210416ac800c7111dec054636c32786bd1ca7d39fda169154ada WatchSource:0}: Error finding container ed2167ed0534210416ac800c7111dec054636c32786bd1ca7d39fda169154ada: Status 404 returned error can't find the container with id ed2167ed0534210416ac800c7111dec054636c32786bd1ca7d39fda169154ada Jul 15 19:17:23.062446 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:23.062412 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gtxhr" event={"ID":"b087f738-59f9-4bcb-b227-33cfdee6adc9","Type":"ContainerStarted","Data":"f929c33ac086fa1ac72911acb2129b2ea928efb90313e0eae543711b4e58bb7a"} Jul 15 19:17:23.064212 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:23.064162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ftxlr" event={"ID":"a6cf0904-b4aa-403b-b630-4281b5bcd835","Type":"ContainerStarted","Data":"ed2167ed0534210416ac800c7111dec054636c32786bd1ca7d39fda169154ada"} Jul 15 19:17:24.071537 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:24.070993 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gtxhr" event={"ID":"b087f738-59f9-4bcb-b227-33cfdee6adc9","Type":"ContainerStarted","Data":"78a38cb6ac6d41d2dccca1721a7ab59b3d84f95681439d40765b9564984276b9"} Jul 15 19:17:24.090590 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:24.090526 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gtxhr" podStartSLOduration=65.997255812 podStartE2EDuration="1m7.090503717s" totalImagesPullingTime="1.093247905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:24.089441962 +0000 UTC m=+89.926146743" watchObservedRunningTime="2026-07-15 19:17:24.090503717 +0000 UTC m=+89.927208495" Jul 15 19:17:27.082629 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:27.082567 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j8jpj" event={"ID":"648cd87d-755f-4648-97d3-0857e63a32bb","Type":"ContainerStarted","Data":"9fb34346c0b094e39feb79092b3533cf0880fd71c4fa1eeaee4648d91602336d"} Jul 15 19:17:27.084269 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:27.084240 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ftxlr" event={"ID":"a6cf0904-b4aa-403b-b630-4281b5bcd835","Type":"ContainerStarted","Data":"6255eee156542b0b73f4ea43fa3b24076aaa14d396d6399eef82c3199287c6a9"} Jul 15 19:17:27.084382 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:27.084333 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:17:27.097675 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:27.097616 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j8jpj" podStartSLOduration=65.427700173 podStartE2EDuration="1m10.09760054s" totalImagesPullingTime="4.669900367s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:27.096701575 +0000 UTC m=+92.933406351" watchObservedRunningTime="2026-07-15 19:17:27.09760054 +0000 UTC m=+92.934305317" Jul 15 19:17:27.110743 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:27.110684 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ftxlr" podStartSLOduration=65.767467649 podStartE2EDuration="1m10.110666222s" totalImagesPullingTime="4.343198573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:17:27.110613402 +0000 UTC m=+92.947318177" watchObservedRunningTime="2026-07-15 19:17:27.110666222 +0000 UTC m=+92.947370999" Jul 15 19:17:58.090601 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:17:58.090548 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ftxlr" Jul 15 19:18:02.386492 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.386457 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:18:02.390095 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.390078 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.392834 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.392811 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5wtsc\"" Jul 15 19:18:02.392962 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.392845 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 19:18:02.392962 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.392892 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 19:18:02.392962 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.392908 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 19:18:02.393117 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.393012 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 19:18:02.393117 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.393056 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 19:18:02.393249 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.393234 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 19:18:02.393314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.393234 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 19:18:02.393314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.393235 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 19:18:02.398263 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.398242 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 19:18:02.404188 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.404167 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:18:02.560508 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560508 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560516 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-config-volume\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560748 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560537 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-web-config\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560748 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c32b9973-d4e6-408f-83cf-699e25887287-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560748 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560690 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32b9973-d4e6-408f-83cf-699e25887287-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560748 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c32b9973-d4e6-408f-83cf-699e25887287-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560748 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560899 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560764 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c32b9973-d4e6-408f-83cf-699e25887287-config-out\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560899 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560806 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c32b9973-d4e6-408f-83cf-699e25887287-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560899 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560984 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560984 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.560984 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.560941 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqpdg\" (UniqueName: \"kubernetes.io/projected/c32b9973-d4e6-408f-83cf-699e25887287-kube-api-access-rqpdg\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.661886 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.661886 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661832 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-config-volume\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.661886 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-web-config\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.661886 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c32b9973-d4e6-408f-83cf-699e25887287-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.661886 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32b9973-d4e6-408f-83cf-699e25887287-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.662244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c32b9973-d4e6-408f-83cf-699e25887287-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.662244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.662244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c32b9973-d4e6-408f-83cf-699e25887287-config-out\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.662244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c32b9973-d4e6-408f-83cf-699e25887287-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.662244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.661985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.662244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.662011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.662244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.662035 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.662244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.662063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqpdg\" (UniqueName: \"kubernetes.io/projected/c32b9973-d4e6-408f-83cf-699e25887287-kube-api-access-rqpdg\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.663337 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.662837 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c32b9973-d4e6-408f-83cf-699e25887287-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.663337 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.663307 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c32b9973-d4e6-408f-83cf-699e25887287-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.663550 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.663435 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c32b9973-d4e6-408f-83cf-699e25887287-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.665388 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.665243 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.665799 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.665694 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c32b9973-d4e6-408f-83cf-699e25887287-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.665799 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.665694 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-config-volume\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.665799 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.665758 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c32b9973-d4e6-408f-83cf-699e25887287-config-out\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.665985 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.665868 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-web-config\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.665985 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.665975 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.666347 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.666325 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.666401 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.666377 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.667014 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.666998 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c32b9973-d4e6-408f-83cf-699e25887287-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.670336 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.670310 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqpdg\" (UniqueName: \"kubernetes.io/projected/c32b9973-d4e6-408f-83cf-699e25887287-kube-api-access-rqpdg\") pod \"alertmanager-main-0\" (UID: \"c32b9973-d4e6-408f-83cf-699e25887287\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.699341 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.699314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:18:02.829803 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:02.829608 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:18:02.832390 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:18:02.832362 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc32b9973_d4e6_408f_83cf_699e25887287.slice/crio-4dbee3d2119449752b637b554959322fbd2019e37b3518738d4123963dc9ec42 WatchSource:0}: Error finding container 4dbee3d2119449752b637b554959322fbd2019e37b3518738d4123963dc9ec42: Status 404 returned error can't find the container with id 4dbee3d2119449752b637b554959322fbd2019e37b3518738d4123963dc9ec42 Jul 15 19:18:03.200420 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:03.200385 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c32b9973-d4e6-408f-83cf-699e25887287","Type":"ContainerStarted","Data":"4dbee3d2119449752b637b554959322fbd2019e37b3518738d4123963dc9ec42"} Jul 15 19:18:04.204965 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.204927 2562 generic.go:356] "Generic (PLEG): container finished" podID="c32b9973-d4e6-408f-83cf-699e25887287" containerID="e6ecf7476ad4ef8033c3ee9cb93fa63d50b48e934680d1cfbdb5117ef956efd1" exitCode=0 Jul 15 19:18:04.205338 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.205004 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c32b9973-d4e6-408f-83cf-699e25887287","Type":"ContainerDied","Data":"e6ecf7476ad4ef8033c3ee9cb93fa63d50b48e934680d1cfbdb5117ef956efd1"} Jul 15 19:18:04.542949 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.542914 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2"] Jul 15 19:18:04.545471 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.545455 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.548248 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.548222 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 19:18:04.548394 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.548260 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 19:18:04.548394 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.548282 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 19:18:04.548394 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.548283 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dg8k2\"" Jul 15 19:18:04.548394 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.548318 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 19:18:04.548394 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.548266 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 19:18:04.553475 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.553458 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 19:18:04.560787 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.560764 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2"] Jul 15 19:18:04.677483 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.677450 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5b4050-a275-46e3-a713-e6cf11b5af5b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.677483 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.677486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5b4050-a275-46e3-a713-e6cf11b5af5b-serving-certs-ca-bundle\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.677696 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.677508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhwxs\" (UniqueName: \"kubernetes.io/projected/6f5b4050-a275-46e3-a713-e6cf11b5af5b-kube-api-access-fhwxs\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.677696 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.677531 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.677696 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.677618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-federate-client-tls\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.677696 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.677643 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-secret-telemeter-client\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.677696 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.677675 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-telemeter-client-tls\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.677844 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.677699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f5b4050-a275-46e3-a713-e6cf11b5af5b-metrics-client-ca\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.779023 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.778992 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5b4050-a275-46e3-a713-e6cf11b5af5b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.779023 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.779026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5b4050-a275-46e3-a713-e6cf11b5af5b-serving-certs-ca-bundle\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.779218 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.779050 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwxs\" (UniqueName: \"kubernetes.io/projected/6f5b4050-a275-46e3-a713-e6cf11b5af5b-kube-api-access-fhwxs\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.779218 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.779073 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.779218 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.779101 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-federate-client-tls\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.779218 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.779117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-secret-telemeter-client\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.779218 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.779138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-telemeter-client-tls\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.779218 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.779156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f5b4050-a275-46e3-a713-e6cf11b5af5b-metrics-client-ca\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.779880 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.779847 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5b4050-a275-46e3-a713-e6cf11b5af5b-serving-certs-ca-bundle\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.780197 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.779969 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f5b4050-a275-46e3-a713-e6cf11b5af5b-metrics-client-ca\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.780513 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.780484 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5b4050-a275-46e3-a713-e6cf11b5af5b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.782018 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.781989 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-telemeter-client-tls\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.782116 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.782030 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-secret-telemeter-client\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.782267 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.782246 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.782336 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.782267 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6f5b4050-a275-46e3-a713-e6cf11b5af5b-federate-client-tls\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.786869 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.786842 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhwxs\" (UniqueName: \"kubernetes.io/projected/6f5b4050-a275-46e3-a713-e6cf11b5af5b-kube-api-access-fhwxs\") pod \"telemeter-client-6f4f5f7f69-gb5x2\" (UID: \"6f5b4050-a275-46e3-a713-e6cf11b5af5b\") " pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.854945 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.854857 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" Jul 15 19:18:04.981875 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:04.981846 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2"] Jul 15 19:18:04.985017 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:18:04.984993 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5b4050_a275_46e3_a713_e6cf11b5af5b.slice/crio-a86dd67240d65e3b0de25a6e52be8f3104bcc5b33e2db1869b9948261baf3e38 WatchSource:0}: Error finding container a86dd67240d65e3b0de25a6e52be8f3104bcc5b33e2db1869b9948261baf3e38: Status 404 returned error can't find the container with id a86dd67240d65e3b0de25a6e52be8f3104bcc5b33e2db1869b9948261baf3e38 Jul 15 19:18:05.210976 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:05.210940 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" event={"ID":"6f5b4050-a275-46e3-a713-e6cf11b5af5b","Type":"ContainerStarted","Data":"a86dd67240d65e3b0de25a6e52be8f3104bcc5b33e2db1869b9948261baf3e38"} Jul 15 19:18:06.217350 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:06.217220 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c32b9973-d4e6-408f-83cf-699e25887287","Type":"ContainerStarted","Data":"c5c29c81303f72bde04fb407d42d03fc49e3bf2b3e36cd253d0676fc7cd97c05"} Jul 15 19:18:06.217350 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:06.217308 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c32b9973-d4e6-408f-83cf-699e25887287","Type":"ContainerStarted","Data":"6f56c721d2d6a4d5b8f3ac50e4acaac7d10953b92f6e62036e5fa68623596e25"} Jul 15 19:18:07.223211 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:07.223176 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c32b9973-d4e6-408f-83cf-699e25887287","Type":"ContainerStarted","Data":"d563250c20b26055c3c3bb78a6dc558ce22b6920326498dac74f3a9037e2c68b"} Jul 15 19:18:07.223643 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:07.223241 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c32b9973-d4e6-408f-83cf-699e25887287","Type":"ContainerStarted","Data":"5a7f4067f1619260e3250d25dcb463a3e3c4ca899f1f33a6f30af34c796cc724"} Jul 15 19:18:07.223643 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:07.223267 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c32b9973-d4e6-408f-83cf-699e25887287","Type":"ContainerStarted","Data":"926b9d5e5ec59e5fa4fd4e30bb8c9086d5d16c69a0a842184b30fefec821f2e3"} Jul 15 19:18:07.224876 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:07.224850 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" event={"ID":"6f5b4050-a275-46e3-a713-e6cf11b5af5b","Type":"ContainerStarted","Data":"dd520187bdc55c49ffff34c28b5e1d895791cb01ebc4471c5df91aa87ce52a83"} Jul 15 19:18:07.224992 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:07.224902 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" event={"ID":"6f5b4050-a275-46e3-a713-e6cf11b5af5b","Type":"ContainerStarted","Data":"95075019b5449ed5ecf3c24d37f973ee7afff0cc3bdb9c58762dc051341cdff5"} Jul 15 19:18:07.224992 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:07.224928 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" event={"ID":"6f5b4050-a275-46e3-a713-e6cf11b5af5b","Type":"ContainerStarted","Data":"bddc345f02b2de8c6b8f4873dfaf837741aea614a12ce6af7c96265bf3fe3a36"} Jul 15 19:18:07.247340 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:07.247294 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6f4f5f7f69-gb5x2" podStartSLOduration=1.425587303 podStartE2EDuration="3.247271508s" totalImagesPullingTime="1.821684205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:18:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:18:07.245269928 +0000 UTC m=+133.081974706" watchObservedRunningTime="2026-07-15 19:18:07.247271508 +0000 UTC m=+133.083976287" Jul 15 19:18:08.230966 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:08.230924 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c32b9973-d4e6-408f-83cf-699e25887287","Type":"ContainerStarted","Data":"36579a58d4dfccd1050f94713a2e6afeadb25cb287dc3ad2914c95049242df1d"} Jul 15 19:18:08.259296 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:18:08.259219 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.304523882 podStartE2EDuration="6.259198715s" totalImagesPullingTime="3.937661739s" totalInitContainerRuntime="17.013094ms" isStatefulPod=false podCreationTimestamp="2026-07-15 19:18:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:18:08.256958253 +0000 UTC m=+134.093663030" watchObservedRunningTime="2026-07-15 19:18:08.259198715 +0000 UTC m=+134.095903494" Jul 15 19:20:45.578619 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.578568 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w"] Jul 15 19:20:45.581781 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.581762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.584468 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.584443 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-twkrt\"" Jul 15 19:20:45.584468 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.584457 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 19:20:45.585624 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.585607 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 19:20:45.590017 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.589996 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w"] Jul 15 19:20:45.673052 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.673014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.673224 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.673060 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.673224 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.673097 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5trbq\" (UniqueName: \"kubernetes.io/projected/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-kube-api-access-5trbq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.773659 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.773619 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.773659 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.773660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.773860 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.773691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5trbq\" (UniqueName: \"kubernetes.io/projected/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-kube-api-access-5trbq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.774043 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.774025 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.774077 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.774056 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.782151 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.782128 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5trbq\" (UniqueName: \"kubernetes.io/projected/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-kube-api-access-5trbq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:45.891305 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:45.891270 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:20:46.024002 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:46.023972 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w"] Jul 15 19:20:46.027061 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:20:46.027026 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5bdfa0_e2ce_4e16_9140_8fe03e63159c.slice/crio-fb390cd8449e80ea6ce762164e228418c07c510f77dfb100d8357c9aac640db4 WatchSource:0}: Error finding container fb390cd8449e80ea6ce762164e228418c07c510f77dfb100d8357c9aac640db4: Status 404 returned error can't find the container with id fb390cd8449e80ea6ce762164e228418c07c510f77dfb100d8357c9aac640db4 Jul 15 19:20:46.744925 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:46.744891 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" event={"ID":"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c","Type":"ContainerStarted","Data":"fb390cd8449e80ea6ce762164e228418c07c510f77dfb100d8357c9aac640db4"} Jul 15 19:20:53.771255 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:53.771168 2562 generic.go:356] "Generic (PLEG): container finished" podID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerID="0971a2566ca403940cb474c049e0d701497a35e7496c92cf87d8ccc75f4066e2" exitCode=0 Jul 15 19:20:53.771255 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:53.771215 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" event={"ID":"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c","Type":"ContainerDied","Data":"0971a2566ca403940cb474c049e0d701497a35e7496c92cf87d8ccc75f4066e2"} Jul 15 19:20:54.562027 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:54.561987 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:20:54.565442 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:54.565417 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:20:54.567528 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:54.567506 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 19:20:55.779892 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:55.779855 2562 generic.go:356] "Generic (PLEG): container finished" podID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerID="9573466350e8e65de3ab54600b45d9f37a98743f566e1384683111b9fde0bc71" exitCode=0 Jul 15 19:20:55.780342 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:55.779943 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" event={"ID":"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c","Type":"ContainerDied","Data":"9573466350e8e65de3ab54600b45d9f37a98743f566e1384683111b9fde0bc71"} Jul 15 19:20:55.781080 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:20:55.781064 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:21:03.812012 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:03.811968 2562 generic.go:356] "Generic (PLEG): container finished" podID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerID="e7321cb5a9c79f2bb24645fff931602fcdd8e837ee1219781cb949689a012cff" exitCode=0 Jul 15 19:21:03.812449 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:03.812062 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" event={"ID":"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c","Type":"ContainerDied","Data":"e7321cb5a9c79f2bb24645fff931602fcdd8e837ee1219781cb949689a012cff"} Jul 15 19:21:04.943740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:04.943716 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:21:05.025213 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.025179 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-util\") pod \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " Jul 15 19:21:05.025213 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.025221 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5trbq\" (UniqueName: \"kubernetes.io/projected/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-kube-api-access-5trbq\") pod \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " Jul 15 19:21:05.025466 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.025253 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-bundle\") pod \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\" (UID: \"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c\") " Jul 15 19:21:05.026004 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.025977 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-bundle" (OuterVolumeSpecName: "bundle") pod "ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" (UID: "ab5bdfa0-e2ce-4e16-9140-8fe03e63159c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:21:05.027607 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.027568 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-kube-api-access-5trbq" (OuterVolumeSpecName: "kube-api-access-5trbq") pod "ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" (UID: "ab5bdfa0-e2ce-4e16-9140-8fe03e63159c"). InnerVolumeSpecName "kube-api-access-5trbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:21:05.030904 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.030876 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-util" (OuterVolumeSpecName: "util") pod "ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" (UID: "ab5bdfa0-e2ce-4e16-9140-8fe03e63159c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:21:05.126710 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.126622 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:21:05.126710 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.126650 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5trbq\" (UniqueName: \"kubernetes.io/projected/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-kube-api-access-5trbq\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:21:05.126710 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.126661 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab5bdfa0-e2ce-4e16-9140-8fe03e63159c-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:21:05.820767 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.820739 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" Jul 15 19:21:05.820767 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.820750 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zwm4w" event={"ID":"ab5bdfa0-e2ce-4e16-9140-8fe03e63159c","Type":"ContainerDied","Data":"fb390cd8449e80ea6ce762164e228418c07c510f77dfb100d8357c9aac640db4"} Jul 15 19:21:05.820978 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:05.820794 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb390cd8449e80ea6ce762164e228418c07c510f77dfb100d8357c9aac640db4" Jul 15 19:21:13.531002 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.530970 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x"] Jul 15 19:21:13.531387 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.531205 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerName="pull" Jul 15 19:21:13.531387 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.531214 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerName="pull" Jul 15 19:21:13.531387 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.531225 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerName="util" Jul 15 19:21:13.531387 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.531230 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerName="util" Jul 15 19:21:13.531387 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.531237 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerName="extract" Jul 15 19:21:13.531387 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.531242 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerName="extract" Jul 15 19:21:13.531387 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.531284 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab5bdfa0-e2ce-4e16-9140-8fe03e63159c" containerName="extract" Jul 15 19:21:13.534816 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.534798 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" Jul 15 19:21:13.537867 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.537835 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:21:13.537979 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.537835 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 19:21:13.537979 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.537951 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zz4nj\"" Jul 15 19:21:13.549801 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.549776 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x"] Jul 15 19:21:13.587163 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.587125 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7895b82d-72ea-4ff4-874c-ef2a60e0f842-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-66l2x\" (UID: \"7895b82d-72ea-4ff4-874c-ef2a60e0f842\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" Jul 15 19:21:13.587351 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.587182 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v72w\" (UniqueName: \"kubernetes.io/projected/7895b82d-72ea-4ff4-874c-ef2a60e0f842-kube-api-access-4v72w\") pod \"cert-manager-operator-controller-manager-55c444bf86-66l2x\" (UID: \"7895b82d-72ea-4ff4-874c-ef2a60e0f842\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" Jul 15 19:21:13.688560 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.688526 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7895b82d-72ea-4ff4-874c-ef2a60e0f842-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-66l2x\" (UID: \"7895b82d-72ea-4ff4-874c-ef2a60e0f842\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" Jul 15 19:21:13.688681 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.688602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v72w\" (UniqueName: \"kubernetes.io/projected/7895b82d-72ea-4ff4-874c-ef2a60e0f842-kube-api-access-4v72w\") pod \"cert-manager-operator-controller-manager-55c444bf86-66l2x\" (UID: \"7895b82d-72ea-4ff4-874c-ef2a60e0f842\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" Jul 15 19:21:13.688938 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.688918 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7895b82d-72ea-4ff4-874c-ef2a60e0f842-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-66l2x\" (UID: \"7895b82d-72ea-4ff4-874c-ef2a60e0f842\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" Jul 15 19:21:13.698377 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.698348 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v72w\" (UniqueName: \"kubernetes.io/projected/7895b82d-72ea-4ff4-874c-ef2a60e0f842-kube-api-access-4v72w\") pod \"cert-manager-operator-controller-manager-55c444bf86-66l2x\" (UID: \"7895b82d-72ea-4ff4-874c-ef2a60e0f842\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" Jul 15 19:21:13.843968 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.843867 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" Jul 15 19:21:13.978314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:13.978293 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x"] Jul 15 19:21:13.980681 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:21:13.980652 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7895b82d_72ea_4ff4_874c_ef2a60e0f842.slice/crio-f9f723d6dafb23dfa653c03fba47a6d5c56051d638a7194e0d3cc8527c160cc5 WatchSource:0}: Error finding container f9f723d6dafb23dfa653c03fba47a6d5c56051d638a7194e0d3cc8527c160cc5: Status 404 returned error can't find the container with id f9f723d6dafb23dfa653c03fba47a6d5c56051d638a7194e0d3cc8527c160cc5 Jul 15 19:21:14.855066 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:14.855016 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" event={"ID":"7895b82d-72ea-4ff4-874c-ef2a60e0f842","Type":"ContainerStarted","Data":"f9f723d6dafb23dfa653c03fba47a6d5c56051d638a7194e0d3cc8527c160cc5"} Jul 15 19:21:15.860504 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:15.860404 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" event={"ID":"7895b82d-72ea-4ff4-874c-ef2a60e0f842","Type":"ContainerStarted","Data":"aff1103150d86e9a2c8c1362df0c9ffe9a6e2a1fd26fc78707d8f57ef2d86864"} Jul 15 19:21:15.882260 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:15.882205 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-66l2x" podStartSLOduration=1.329434653 podStartE2EDuration="2.882187429s" totalImagesPullingTime="1.552752776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:15.880507335 +0000 UTC m=+321.717212105" watchObservedRunningTime="2026-07-15 19:21:15.882187429 +0000 UTC m=+321.718892207" Jul 15 19:21:17.268904 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.268870 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k"] Jul 15 19:21:17.271344 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.271320 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.274032 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.274010 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 19:21:17.274165 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.274014 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-twkrt\"" Jul 15 19:21:17.275308 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.275291 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 19:21:17.280746 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.280725 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k"] Jul 15 19:21:17.415788 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.415758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.415962 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.415800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.415962 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.415880 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rkfq\" (UniqueName: \"kubernetes.io/projected/da52c00c-f153-46ee-a6c4-6fae9b121769-kube-api-access-6rkfq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.517230 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.517189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.517395 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.517243 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.517395 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.517293 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rkfq\" (UniqueName: \"kubernetes.io/projected/da52c00c-f153-46ee-a6c4-6fae9b121769-kube-api-access-6rkfq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.517704 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.517675 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.517704 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.517687 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.532494 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.532412 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-94ngs"] Jul 15 19:21:17.534243 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.534217 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rkfq\" (UniqueName: \"kubernetes.io/projected/da52c00c-f153-46ee-a6c4-6fae9b121769-kube-api-access-6rkfq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.534518 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.534501 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:17.537146 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.537128 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 19:21:17.537267 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.537151 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-gzdcq\"" Jul 15 19:21:17.537267 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.537219 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 19:21:17.544847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.544820 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-94ngs"] Jul 15 19:21:17.579944 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.579907 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:17.618526 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.618488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32de8b65-4a18-4643-a8f5-6aa341d5f0b2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-94ngs\" (UID: \"32de8b65-4a18-4643-a8f5-6aa341d5f0b2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:17.618714 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.618540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz9wv\" (UniqueName: \"kubernetes.io/projected/32de8b65-4a18-4643-a8f5-6aa341d5f0b2-kube-api-access-kz9wv\") pod \"cert-manager-webhook-6486ffd997-94ngs\" (UID: \"32de8b65-4a18-4643-a8f5-6aa341d5f0b2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:17.709083 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.709054 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k"] Jul 15 19:21:17.711740 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:21:17.711713 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda52c00c_f153_46ee_a6c4_6fae9b121769.slice/crio-214537dd75a44f01f304414fc818c1d51ca20f9069b98327df036b99010cb711 WatchSource:0}: Error finding container 214537dd75a44f01f304414fc818c1d51ca20f9069b98327df036b99010cb711: Status 404 returned error can't find the container with id 214537dd75a44f01f304414fc818c1d51ca20f9069b98327df036b99010cb711 Jul 15 19:21:17.719170 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.719153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32de8b65-4a18-4643-a8f5-6aa341d5f0b2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-94ngs\" (UID: \"32de8b65-4a18-4643-a8f5-6aa341d5f0b2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:17.719236 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.719183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz9wv\" (UniqueName: \"kubernetes.io/projected/32de8b65-4a18-4643-a8f5-6aa341d5f0b2-kube-api-access-kz9wv\") pod \"cert-manager-webhook-6486ffd997-94ngs\" (UID: \"32de8b65-4a18-4643-a8f5-6aa341d5f0b2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:17.728562 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.728540 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz9wv\" (UniqueName: \"kubernetes.io/projected/32de8b65-4a18-4643-a8f5-6aa341d5f0b2-kube-api-access-kz9wv\") pod \"cert-manager-webhook-6486ffd997-94ngs\" (UID: \"32de8b65-4a18-4643-a8f5-6aa341d5f0b2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:17.728562 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.728569 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/32de8b65-4a18-4643-a8f5-6aa341d5f0b2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-94ngs\" (UID: \"32de8b65-4a18-4643-a8f5-6aa341d5f0b2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:17.858122 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.858088 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:17.869548 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.869441 2562 generic.go:356] "Generic (PLEG): container finished" podID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerID="8d4f4064a560d69740851b1facf3082e2cafb665e0666257b534a3e2a6086377" exitCode=0 Jul 15 19:21:17.869548 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.869533 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" event={"ID":"da52c00c-f153-46ee-a6c4-6fae9b121769","Type":"ContainerDied","Data":"8d4f4064a560d69740851b1facf3082e2cafb665e0666257b534a3e2a6086377"} Jul 15 19:21:17.869748 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.869601 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" event={"ID":"da52c00c-f153-46ee-a6c4-6fae9b121769","Type":"ContainerStarted","Data":"214537dd75a44f01f304414fc818c1d51ca20f9069b98327df036b99010cb711"} Jul 15 19:21:17.984069 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:17.984045 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-94ngs"] Jul 15 19:21:17.986120 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:21:17.986092 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32de8b65_4a18_4643_a8f5_6aa341d5f0b2.slice/crio-f609bb3c37b78886c8dcac4a88f0c83b2cd695136adfcf152f63b4c0ce585991 WatchSource:0}: Error finding container f609bb3c37b78886c8dcac4a88f0c83b2cd695136adfcf152f63b4c0ce585991: Status 404 returned error can't find the container with id f609bb3c37b78886c8dcac4a88f0c83b2cd695136adfcf152f63b4c0ce585991 Jul 15 19:21:18.876187 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:18.876093 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" event={"ID":"32de8b65-4a18-4643-a8f5-6aa341d5f0b2","Type":"ContainerStarted","Data":"f609bb3c37b78886c8dcac4a88f0c83b2cd695136adfcf152f63b4c0ce585991"} Jul 15 19:21:19.965332 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:19.965296 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj"] Jul 15 19:21:19.970553 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:19.970170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" Jul 15 19:21:19.973420 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:19.973391 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-g7qxf\"" Jul 15 19:21:19.981322 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:19.981284 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj"] Jul 15 19:21:20.038842 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:20.038809 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2zsb\" (UniqueName: \"kubernetes.io/projected/13b4e50d-57d1-4244-9d4d-23d43b88652d-kube-api-access-x2zsb\") pod \"cert-manager-cainjector-559cdd8c87-rqqvj\" (UID: \"13b4e50d-57d1-4244-9d4d-23d43b88652d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" Jul 15 19:21:20.038984 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:20.038865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13b4e50d-57d1-4244-9d4d-23d43b88652d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rqqvj\" (UID: \"13b4e50d-57d1-4244-9d4d-23d43b88652d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" Jul 15 19:21:20.139977 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:20.139929 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2zsb\" (UniqueName: \"kubernetes.io/projected/13b4e50d-57d1-4244-9d4d-23d43b88652d-kube-api-access-x2zsb\") pod \"cert-manager-cainjector-559cdd8c87-rqqvj\" (UID: \"13b4e50d-57d1-4244-9d4d-23d43b88652d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" Jul 15 19:21:20.140171 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:20.140005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13b4e50d-57d1-4244-9d4d-23d43b88652d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rqqvj\" (UID: \"13b4e50d-57d1-4244-9d4d-23d43b88652d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" Jul 15 19:21:20.148801 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:20.148773 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13b4e50d-57d1-4244-9d4d-23d43b88652d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rqqvj\" (UID: \"13b4e50d-57d1-4244-9d4d-23d43b88652d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" Jul 15 19:21:20.148980 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:20.148962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2zsb\" (UniqueName: \"kubernetes.io/projected/13b4e50d-57d1-4244-9d4d-23d43b88652d-kube-api-access-x2zsb\") pod \"cert-manager-cainjector-559cdd8c87-rqqvj\" (UID: \"13b4e50d-57d1-4244-9d4d-23d43b88652d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" Jul 15 19:21:20.280166 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:20.280071 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" Jul 15 19:21:21.094743 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:21.094714 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj"] Jul 15 19:21:21.100758 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:21:21.100723 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13b4e50d_57d1_4244_9d4d_23d43b88652d.slice/crio-86a24f6072446eb1f804c6cab812da04e5ea548058cfd4270fe8527748ebb444 WatchSource:0}: Error finding container 86a24f6072446eb1f804c6cab812da04e5ea548058cfd4270fe8527748ebb444: Status 404 returned error can't find the container with id 86a24f6072446eb1f804c6cab812da04e5ea548058cfd4270fe8527748ebb444 Jul 15 19:21:21.890793 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:21.890751 2562 generic.go:356] "Generic (PLEG): container finished" podID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerID="c14061bbee1aaee7838f4f59d2a6b74abc44d6c7da95e5b1845f908f85f522aa" exitCode=0 Jul 15 19:21:21.890966 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:21.890831 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" event={"ID":"da52c00c-f153-46ee-a6c4-6fae9b121769","Type":"ContainerDied","Data":"c14061bbee1aaee7838f4f59d2a6b74abc44d6c7da95e5b1845f908f85f522aa"} Jul 15 19:21:21.892557 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:21.892532 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" event={"ID":"32de8b65-4a18-4643-a8f5-6aa341d5f0b2","Type":"ContainerStarted","Data":"fa3f114cf08f64e6457c7da351ae091f519125fbae04cd95e5408d699d69b341"} Jul 15 19:21:21.892702 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:21.892626 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:21.894000 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:21.893981 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" event={"ID":"13b4e50d-57d1-4244-9d4d-23d43b88652d","Type":"ContainerStarted","Data":"e92a27f81e2b74d2ba65c028b1a952af3da8d82edf3f2b61c03a7fa17a309f3c"} Jul 15 19:21:21.894068 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:21.894010 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" event={"ID":"13b4e50d-57d1-4244-9d4d-23d43b88652d","Type":"ContainerStarted","Data":"86a24f6072446eb1f804c6cab812da04e5ea548058cfd4270fe8527748ebb444"} Jul 15 19:21:21.923749 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:21.923703 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" podStartSLOduration=1.948779418 podStartE2EDuration="4.923688678s" totalImagesPullingTime="2.97490926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:21.922665875 +0000 UTC m=+327.759370653" watchObservedRunningTime="2026-07-15 19:21:21.923688678 +0000 UTC m=+327.760393452" Jul 15 19:21:21.938381 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:21.938335 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqqvj" podStartSLOduration=2.9383196529999998 podStartE2EDuration="2.938319653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:21.936897236 +0000 UTC m=+327.773602013" watchObservedRunningTime="2026-07-15 19:21:21.938319653 +0000 UTC m=+327.775024431" Jul 15 19:21:22.899614 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:22.899563 2562 generic.go:356] "Generic (PLEG): container finished" podID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerID="baf2a0b6e1c9c78b73794830cee73e49b0303213b6974d03fda44ca61dc681c9" exitCode=0 Jul 15 19:21:22.900009 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:22.899660 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" event={"ID":"da52c00c-f153-46ee-a6c4-6fae9b121769","Type":"ContainerDied","Data":"baf2a0b6e1c9c78b73794830cee73e49b0303213b6974d03fda44ca61dc681c9"} Jul 15 19:21:24.026306 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.026278 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:24.170958 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.170865 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-bundle\") pod \"da52c00c-f153-46ee-a6c4-6fae9b121769\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " Jul 15 19:21:24.170958 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.170931 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-util\") pod \"da52c00c-f153-46ee-a6c4-6fae9b121769\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " Jul 15 19:21:24.171178 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.170981 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rkfq\" (UniqueName: \"kubernetes.io/projected/da52c00c-f153-46ee-a6c4-6fae9b121769-kube-api-access-6rkfq\") pod \"da52c00c-f153-46ee-a6c4-6fae9b121769\" (UID: \"da52c00c-f153-46ee-a6c4-6fae9b121769\") " Jul 15 19:21:24.171319 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.171288 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-bundle" (OuterVolumeSpecName: "bundle") pod "da52c00c-f153-46ee-a6c4-6fae9b121769" (UID: "da52c00c-f153-46ee-a6c4-6fae9b121769"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:21:24.173709 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.173673 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da52c00c-f153-46ee-a6c4-6fae9b121769-kube-api-access-6rkfq" (OuterVolumeSpecName: "kube-api-access-6rkfq") pod "da52c00c-f153-46ee-a6c4-6fae9b121769" (UID: "da52c00c-f153-46ee-a6c4-6fae9b121769"). InnerVolumeSpecName "kube-api-access-6rkfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:21:24.175609 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.175563 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-util" (OuterVolumeSpecName: "util") pod "da52c00c-f153-46ee-a6c4-6fae9b121769" (UID: "da52c00c-f153-46ee-a6c4-6fae9b121769"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:21:24.272165 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.272127 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rkfq\" (UniqueName: \"kubernetes.io/projected/da52c00c-f153-46ee-a6c4-6fae9b121769-kube-api-access-6rkfq\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:21:24.272165 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.272159 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:21:24.272165 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.272169 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da52c00c-f153-46ee-a6c4-6fae9b121769-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:21:24.908763 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.908722 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" event={"ID":"da52c00c-f153-46ee-a6c4-6fae9b121769","Type":"ContainerDied","Data":"214537dd75a44f01f304414fc818c1d51ca20f9069b98327df036b99010cb711"} Jul 15 19:21:24.908763 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.908759 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvc76k" Jul 15 19:21:24.908959 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:24.908776 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="214537dd75a44f01f304414fc818c1d51ca20f9069b98327df036b99010cb711" Jul 15 19:21:27.901893 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:27.901859 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-94ngs" Jul 15 19:21:28.800610 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.800562 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d7h4j"] Jul 15 19:21:28.800834 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.800821 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerName="extract" Jul 15 19:21:28.800884 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.800836 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerName="extract" Jul 15 19:21:28.800884 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.800846 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerName="util" Jul 15 19:21:28.800884 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.800852 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerName="util" Jul 15 19:21:28.800884 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.800869 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerName="pull" Jul 15 19:21:28.800884 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.800874 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerName="pull" Jul 15 19:21:28.801024 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.800916 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="da52c00c-f153-46ee-a6c4-6fae9b121769" containerName="extract" Jul 15 19:21:28.803018 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.803003 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-d7h4j" Jul 15 19:21:28.805665 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.805636 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jtmvf\"" Jul 15 19:21:28.814328 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.814306 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d7h4j"] Jul 15 19:21:28.906764 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.906729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07926de7-5184-42f8-942a-984ec89b6123-bound-sa-token\") pod \"cert-manager-55b4d954d5-d7h4j\" (UID: \"07926de7-5184-42f8-942a-984ec89b6123\") " pod="cert-manager/cert-manager-55b4d954d5-d7h4j" Jul 15 19:21:28.907172 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:28.906786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbwh2\" (UniqueName: \"kubernetes.io/projected/07926de7-5184-42f8-942a-984ec89b6123-kube-api-access-gbwh2\") pod \"cert-manager-55b4d954d5-d7h4j\" (UID: \"07926de7-5184-42f8-942a-984ec89b6123\") " pod="cert-manager/cert-manager-55b4d954d5-d7h4j" Jul 15 19:21:29.007208 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:29.007176 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07926de7-5184-42f8-942a-984ec89b6123-bound-sa-token\") pod \"cert-manager-55b4d954d5-d7h4j\" (UID: \"07926de7-5184-42f8-942a-984ec89b6123\") " pod="cert-manager/cert-manager-55b4d954d5-d7h4j" Jul 15 19:21:29.007367 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:29.007229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwh2\" (UniqueName: \"kubernetes.io/projected/07926de7-5184-42f8-942a-984ec89b6123-kube-api-access-gbwh2\") pod \"cert-manager-55b4d954d5-d7h4j\" (UID: \"07926de7-5184-42f8-942a-984ec89b6123\") " pod="cert-manager/cert-manager-55b4d954d5-d7h4j" Jul 15 19:21:29.017432 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:29.017405 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07926de7-5184-42f8-942a-984ec89b6123-bound-sa-token\") pod \"cert-manager-55b4d954d5-d7h4j\" (UID: \"07926de7-5184-42f8-942a-984ec89b6123\") " pod="cert-manager/cert-manager-55b4d954d5-d7h4j" Jul 15 19:21:29.017571 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:29.017411 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbwh2\" (UniqueName: \"kubernetes.io/projected/07926de7-5184-42f8-942a-984ec89b6123-kube-api-access-gbwh2\") pod \"cert-manager-55b4d954d5-d7h4j\" (UID: \"07926de7-5184-42f8-942a-984ec89b6123\") " pod="cert-manager/cert-manager-55b4d954d5-d7h4j" Jul 15 19:21:29.111871 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:29.111769 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-d7h4j" Jul 15 19:21:29.240552 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:29.240468 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-d7h4j"] Jul 15 19:21:29.243193 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:21:29.243156 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07926de7_5184_42f8_942a_984ec89b6123.slice/crio-881d89eb62b96a69742116b3475cf2239e9164895110b801b7083669d4f35850 WatchSource:0}: Error finding container 881d89eb62b96a69742116b3475cf2239e9164895110b801b7083669d4f35850: Status 404 returned error can't find the container with id 881d89eb62b96a69742116b3475cf2239e9164895110b801b7083669d4f35850 Jul 15 19:21:29.929185 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:29.929146 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-d7h4j" event={"ID":"07926de7-5184-42f8-942a-984ec89b6123","Type":"ContainerStarted","Data":"7cd92d50ad88f1cb1d5986e87cd984e9862caac12cda0f5e86d775c68843d038"} Jul 15 19:21:29.929565 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:29.929195 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-d7h4j" event={"ID":"07926de7-5184-42f8-942a-984ec89b6123","Type":"ContainerStarted","Data":"881d89eb62b96a69742116b3475cf2239e9164895110b801b7083669d4f35850"} Jul 15 19:21:29.946342 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:29.946290 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-d7h4j" podStartSLOduration=1.946273755 podStartE2EDuration="1.946273755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:29.94475505 +0000 UTC m=+335.781459829" watchObservedRunningTime="2026-07-15 19:21:29.946273755 +0000 UTC m=+335.782978533" Jul 15 19:21:40.110807 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.110764 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh"] Jul 15 19:21:40.113037 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.113019 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" Jul 15 19:21:40.115671 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.115645 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:21:40.115797 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.115742 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 19:21:40.116962 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.116948 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-n2mp9\"" Jul 15 19:21:40.123344 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.123311 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh"] Jul 15 19:21:40.295221 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.295185 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8cgr\" (UniqueName: \"kubernetes.io/projected/c3d40aa3-be12-4bfa-b115-9211dbd18db6-kube-api-access-n8cgr\") pod \"openshift-lws-operator-fd8ccff4c-tfknh\" (UID: \"c3d40aa3-be12-4bfa-b115-9211dbd18db6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" Jul 15 19:21:40.295221 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.295228 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3d40aa3-be12-4bfa-b115-9211dbd18db6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tfknh\" (UID: \"c3d40aa3-be12-4bfa-b115-9211dbd18db6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" Jul 15 19:21:40.396092 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.396063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3d40aa3-be12-4bfa-b115-9211dbd18db6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tfknh\" (UID: \"c3d40aa3-be12-4bfa-b115-9211dbd18db6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" Jul 15 19:21:40.396246 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.396144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8cgr\" (UniqueName: \"kubernetes.io/projected/c3d40aa3-be12-4bfa-b115-9211dbd18db6-kube-api-access-n8cgr\") pod \"openshift-lws-operator-fd8ccff4c-tfknh\" (UID: \"c3d40aa3-be12-4bfa-b115-9211dbd18db6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" Jul 15 19:21:40.396488 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.396468 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3d40aa3-be12-4bfa-b115-9211dbd18db6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tfknh\" (UID: \"c3d40aa3-be12-4bfa-b115-9211dbd18db6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" Jul 15 19:21:40.408755 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.408723 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8cgr\" (UniqueName: \"kubernetes.io/projected/c3d40aa3-be12-4bfa-b115-9211dbd18db6-kube-api-access-n8cgr\") pod \"openshift-lws-operator-fd8ccff4c-tfknh\" (UID: \"c3d40aa3-be12-4bfa-b115-9211dbd18db6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" Jul 15 19:21:40.422713 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.422691 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" Jul 15 19:21:40.557387 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.557226 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh"] Jul 15 19:21:40.560060 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:21:40.560017 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3d40aa3_be12_4bfa_b115_9211dbd18db6.slice/crio-497eb8c5d8b5ea83ffc753da4896e13117cc06d22be051e4afc051db3309e70b WatchSource:0}: Error finding container 497eb8c5d8b5ea83ffc753da4896e13117cc06d22be051e4afc051db3309e70b: Status 404 returned error can't find the container with id 497eb8c5d8b5ea83ffc753da4896e13117cc06d22be051e4afc051db3309e70b Jul 15 19:21:40.971938 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:21:40.971903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" event={"ID":"c3d40aa3-be12-4bfa-b115-9211dbd18db6","Type":"ContainerStarted","Data":"497eb8c5d8b5ea83ffc753da4896e13117cc06d22be051e4afc051db3309e70b"} Jul 15 19:22:00.047901 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:00.047868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" event={"ID":"c3d40aa3-be12-4bfa-b115-9211dbd18db6","Type":"ContainerStarted","Data":"f0f8ee78b05681a18378dd7c450489de4ca53bd8f9cb1e9f8f41160ba77035bf"} Jul 15 19:22:00.065096 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:00.065002 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tfknh" podStartSLOduration=1.205464032 podStartE2EDuration="20.064986213s" totalImagesPullingTime="18.859522181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:00.063468096 +0000 UTC m=+365.900172875" watchObservedRunningTime="2026-07-15 19:22:00.064986213 +0000 UTC m=+365.901690990" Jul 15 19:22:01.641010 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.640979 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb"] Jul 15 19:22:01.643353 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.643329 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.646345 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.646326 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 19:22:01.646418 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.646358 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 19:22:01.646418 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.646410 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-twkrt\"" Jul 15 19:22:01.652943 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.652920 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb"] Jul 15 19:22:01.750219 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.750187 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.750407 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.750239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97dbb\" (UniqueName: \"kubernetes.io/projected/f7f009c0-8055-4889-85e0-37742df2cf5a-kube-api-access-97dbb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.750407 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.750313 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.851190 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.851146 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.851378 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.851216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97dbb\" (UniqueName: \"kubernetes.io/projected/f7f009c0-8055-4889-85e0-37742df2cf5a-kube-api-access-97dbb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.851378 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.851262 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.851547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.851527 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.851664 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.851645 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.860203 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.860167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97dbb\" (UniqueName: \"kubernetes.io/projected/f7f009c0-8055-4889-85e0-37742df2cf5a-kube-api-access-97dbb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:01.952282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:01.952196 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:02.081116 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:02.081089 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb"] Jul 15 19:22:02.083529 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:22:02.083498 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f009c0_8055_4889_85e0_37742df2cf5a.slice/crio-fc8b0fb517c4807f10baac3f2b3ac77dc58b4b5de303aae6fd021c68f2bf11f1 WatchSource:0}: Error finding container fc8b0fb517c4807f10baac3f2b3ac77dc58b4b5de303aae6fd021c68f2bf11f1: Status 404 returned error can't find the container with id fc8b0fb517c4807f10baac3f2b3ac77dc58b4b5de303aae6fd021c68f2bf11f1 Jul 15 19:22:03.063283 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:03.063248 2562 generic.go:356] "Generic (PLEG): container finished" podID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerID="4f2fba1b916c31e35ebab434ab1eba277e262e6d25d3a30566b477bf906b0122" exitCode=0 Jul 15 19:22:03.063777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:03.063293 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" event={"ID":"f7f009c0-8055-4889-85e0-37742df2cf5a","Type":"ContainerDied","Data":"4f2fba1b916c31e35ebab434ab1eba277e262e6d25d3a30566b477bf906b0122"} Jul 15 19:22:03.063777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:03.063353 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" event={"ID":"f7f009c0-8055-4889-85e0-37742df2cf5a","Type":"ContainerStarted","Data":"fc8b0fb517c4807f10baac3f2b3ac77dc58b4b5de303aae6fd021c68f2bf11f1"} Jul 15 19:22:04.068824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:04.068734 2562 generic.go:356] "Generic (PLEG): container finished" podID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerID="dd99c1b1f9981a8538d42f41be521b0e32a2d3a5734d2c4e39f90e8358ee5061" exitCode=0 Jul 15 19:22:04.068824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:04.068807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" event={"ID":"f7f009c0-8055-4889-85e0-37742df2cf5a","Type":"ContainerDied","Data":"dd99c1b1f9981a8538d42f41be521b0e32a2d3a5734d2c4e39f90e8358ee5061"} Jul 15 19:22:05.074719 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:05.074675 2562 generic.go:356] "Generic (PLEG): container finished" podID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerID="45a62ad820106388d63ee464819958bc4314769f71ce79d8249b02b8adda671c" exitCode=0 Jul 15 19:22:05.075110 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:05.074757 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" event={"ID":"f7f009c0-8055-4889-85e0-37742df2cf5a","Type":"ContainerDied","Data":"45a62ad820106388d63ee464819958bc4314769f71ce79d8249b02b8adda671c"} Jul 15 19:22:06.201102 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.201079 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:06.283374 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.283333 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97dbb\" (UniqueName: \"kubernetes.io/projected/f7f009c0-8055-4889-85e0-37742df2cf5a-kube-api-access-97dbb\") pod \"f7f009c0-8055-4889-85e0-37742df2cf5a\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " Jul 15 19:22:06.283521 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.283391 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-util\") pod \"f7f009c0-8055-4889-85e0-37742df2cf5a\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " Jul 15 19:22:06.283521 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.283433 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-bundle\") pod \"f7f009c0-8055-4889-85e0-37742df2cf5a\" (UID: \"f7f009c0-8055-4889-85e0-37742df2cf5a\") " Jul 15 19:22:06.284162 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.284124 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-bundle" (OuterVolumeSpecName: "bundle") pod "f7f009c0-8055-4889-85e0-37742df2cf5a" (UID: "f7f009c0-8055-4889-85e0-37742df2cf5a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:22:06.285664 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.285639 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f009c0-8055-4889-85e0-37742df2cf5a-kube-api-access-97dbb" (OuterVolumeSpecName: "kube-api-access-97dbb") pod "f7f009c0-8055-4889-85e0-37742df2cf5a" (UID: "f7f009c0-8055-4889-85e0-37742df2cf5a"). InnerVolumeSpecName "kube-api-access-97dbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:22:06.289095 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.289074 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-util" (OuterVolumeSpecName: "util") pod "f7f009c0-8055-4889-85e0-37742df2cf5a" (UID: "f7f009c0-8055-4889-85e0-37742df2cf5a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:22:06.384239 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.384212 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:22:06.384239 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.384237 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97dbb\" (UniqueName: \"kubernetes.io/projected/f7f009c0-8055-4889-85e0-37742df2cf5a-kube-api-access-97dbb\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:22:06.384416 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.384246 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7f009c0-8055-4889-85e0-37742df2cf5a-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:22:06.768085 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.768053 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn"] Jul 15 19:22:06.768310 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.768299 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerName="util" Jul 15 19:22:06.768347 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.768312 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerName="util" Jul 15 19:22:06.768347 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.768324 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerName="pull" Jul 15 19:22:06.768347 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.768331 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerName="pull" Jul 15 19:22:06.768347 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.768341 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerName="extract" Jul 15 19:22:06.768347 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.768346 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerName="extract" Jul 15 19:22:06.768487 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.768385 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7f009c0-8055-4889-85e0-37742df2cf5a" containerName="extract" Jul 15 19:22:06.770522 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.770500 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.773707 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.773689 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 19:22:06.774808 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.774787 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 19:22:06.775060 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.775030 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 19:22:06.775146 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.775086 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8b9nr\"" Jul 15 19:22:06.786411 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.786388 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn"] Jul 15 19:22:06.787129 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.787107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-metrics-cert\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.787219 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.787139 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-manager-config\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.787219 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.787166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w7ph\" (UniqueName: \"kubernetes.io/projected/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-kube-api-access-6w7ph\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.787219 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.787193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-cert\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.888502 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.888469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w7ph\" (UniqueName: \"kubernetes.io/projected/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-kube-api-access-6w7ph\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.888502 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.888508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-cert\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.888795 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.888568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-metrics-cert\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.888795 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.888616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-manager-config\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.889299 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.889273 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-manager-config\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.891314 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.891295 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-metrics-cert\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.891514 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.891490 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-cert\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:06.898105 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:06.898080 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w7ph\" (UniqueName: \"kubernetes.io/projected/ffcaa03a-3d64-4bef-be87-2c155f88e9c6-kube-api-access-6w7ph\") pod \"lws-controller-manager-6c74497d4c-x7mfn\" (UID: \"ffcaa03a-3d64-4bef-be87-2c155f88e9c6\") " pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:07.079103 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:07.079008 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:07.084369 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:07.084342 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" Jul 15 19:22:07.084505 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:07.084367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkzhhb" event={"ID":"f7f009c0-8055-4889-85e0-37742df2cf5a","Type":"ContainerDied","Data":"fc8b0fb517c4807f10baac3f2b3ac77dc58b4b5de303aae6fd021c68f2bf11f1"} Jul 15 19:22:07.084505 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:07.084408 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc8b0fb517c4807f10baac3f2b3ac77dc58b4b5de303aae6fd021c68f2bf11f1" Jul 15 19:22:07.214793 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:07.214770 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn"] Jul 15 19:22:07.216899 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:22:07.216875 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffcaa03a_3d64_4bef_be87_2c155f88e9c6.slice/crio-7e2dae849a73efc6202bb77872026ed1ba7cdf35544dc38ce81b343a91d1d76f WatchSource:0}: Error finding container 7e2dae849a73efc6202bb77872026ed1ba7cdf35544dc38ce81b343a91d1d76f: Status 404 returned error can't find the container with id 7e2dae849a73efc6202bb77872026ed1ba7cdf35544dc38ce81b343a91d1d76f Jul 15 19:22:08.090287 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:08.090256 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" event={"ID":"ffcaa03a-3d64-4bef-be87-2c155f88e9c6","Type":"ContainerStarted","Data":"7e2dae849a73efc6202bb77872026ed1ba7cdf35544dc38ce81b343a91d1d76f"} Jul 15 19:22:09.096346 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:09.096303 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" event={"ID":"ffcaa03a-3d64-4bef-be87-2c155f88e9c6","Type":"ContainerStarted","Data":"8d19f85c3357841a0730a17f8a9422cb0f559e361132743b3c9578ed995d4cb3"} Jul 15 19:22:09.096728 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:09.096449 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:09.115153 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:09.115104 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" podStartSLOduration=1.9536911049999999 podStartE2EDuration="3.115087241s" totalImagesPullingTime="1.161396136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:09.113352275 +0000 UTC m=+374.950057066" watchObservedRunningTime="2026-07-15 19:22:09.115087241 +0000 UTC m=+374.951792019" Jul 15 19:22:18.503668 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.503629 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm"] Jul 15 19:22:18.507769 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.507739 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.512657 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.512634 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 19:22:18.512940 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.512924 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 19:22:18.513250 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.513227 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 19:22:18.513342 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.513308 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-glh4z\"" Jul 15 19:22:18.513435 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.513420 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 19:22:18.527034 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.527007 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm"] Jul 15 19:22:18.582620 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.582565 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-manifests\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.582620 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.582633 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-tmp\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.582843 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.582664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.582843 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.582692 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.582843 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.582822 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tpb4\" (UniqueName: \"kubernetes.io/projected/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-kube-api-access-6tpb4\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.683922 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.683892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-tmp\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.683922 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.683929 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.684180 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.683960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.684180 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.684000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpb4\" (UniqueName: \"kubernetes.io/projected/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-kube-api-access-6tpb4\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.684180 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.684040 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-manifests\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.684400 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.684368 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-manifests\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.686456 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.686426 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-tmp\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.686691 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.686673 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.686743 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.686715 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.693978 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.693956 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tpb4\" (UniqueName: \"kubernetes.io/projected/1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd-kube-api-access-6tpb4\") pod \"opendatahub-operator-controller-manager-f46f5478c-nfgbm\" (UID: \"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd\") " pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.819338 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.819257 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:18.961358 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:18.961286 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm"] Jul 15 19:22:18.963438 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:22:18.963408 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb5b76a_c9df_4ed0_ae2b_7b32516ed9bd.slice/crio-001421b67bf5c1dd13618f3391c7c3bddf7598689e33202b60a38d1bd4083532 WatchSource:0}: Error finding container 001421b67bf5c1dd13618f3391c7c3bddf7598689e33202b60a38d1bd4083532: Status 404 returned error can't find the container with id 001421b67bf5c1dd13618f3391c7c3bddf7598689e33202b60a38d1bd4083532 Jul 15 19:22:19.138966 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:19.138933 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" event={"ID":"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd","Type":"ContainerStarted","Data":"001421b67bf5c1dd13618f3391c7c3bddf7598689e33202b60a38d1bd4083532"} Jul 15 19:22:20.103353 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:20.103314 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c74497d4c-x7mfn" Jul 15 19:22:22.155708 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:22.155670 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" event={"ID":"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd","Type":"ContainerStarted","Data":"7d23e183944d169ae8a0414e7896c75b1a87b00f87cd865bf638f5288e8ee3cb"} Jul 15 19:22:23.161768 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:23.161737 2562 generic.go:356] "Generic (PLEG): container finished" podID="1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd" containerID="7d23e183944d169ae8a0414e7896c75b1a87b00f87cd865bf638f5288e8ee3cb" exitCode=0 Jul 15 19:22:23.162117 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:23.161829 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" event={"ID":"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd","Type":"ContainerDied","Data":"7d23e183944d169ae8a0414e7896c75b1a87b00f87cd865bf638f5288e8ee3cb"} Jul 15 19:22:24.167852 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:24.167817 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" event={"ID":"1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd","Type":"ContainerStarted","Data":"0dbc03c981cf2d0f232367ce672a36e9a6370abff689139cf78cd75c6cdb4715"} Jul 15 19:22:24.168239 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:24.167954 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:24.189975 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:24.189919 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" podStartSLOduration=1.8051974560000001 podStartE2EDuration="6.189899912s" totalImagesPullingTime="3.340389738s" totalInitContainerRuntime="1.044312718s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:24.188242534 +0000 UTC m=+390.024947325" watchObservedRunningTime="2026-07-15 19:22:24.189899912 +0000 UTC m=+390.026604691" Jul 15 19:22:35.174353 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:35.174322 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f46f5478c-nfgbm" Jul 15 19:22:54.359657 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.359617 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj"] Jul 15 19:22:54.362349 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.362325 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.365093 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.365070 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 19:22:54.365213 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.365131 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 19:22:54.365278 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.365248 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 19:22:54.365427 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.365411 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fh9lj\"" Jul 15 19:22:54.375165 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.375144 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj"] Jul 15 19:22:54.392440 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.392417 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.392440 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.392446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.392628 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.392463 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.392628 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.392497 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.392628 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.392537 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.392628 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.392552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.392628 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.392603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z77hq\" (UniqueName: \"kubernetes.io/projected/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-kube-api-access-z77hq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.392628 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.392625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.392836 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.392645 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493307 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493307 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493306 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493524 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493328 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493524 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493524 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493422 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493524 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493449 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493524 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493486 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z77hq\" (UniqueName: \"kubernetes.io/projected/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-kube-api-access-z77hq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493524 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493838 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493895 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493866 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.493895 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.493880 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.494152 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.494067 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.494152 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.494108 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.494152 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.494203 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.496006 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.495977 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.496439 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.496416 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.514460 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.514420 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z77hq\" (UniqueName: \"kubernetes.io/projected/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-kube-api-access-z77hq\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.515608 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.515566 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gjpgj\" (UID: \"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.676093 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.676061 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fh9lj\"" Jul 15 19:22:54.683427 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.683391 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:54.825597 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:54.825546 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj"] Jul 15 19:22:54.827439 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:22:54.827411 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba6bcc4b_f4dd_4141_827a_bc9ea8e7b030.slice/crio-1e2c73eb051610e98052d9e632114c64452dfc74319ec423b9c72ec232f46565 WatchSource:0}: Error finding container 1e2c73eb051610e98052d9e632114c64452dfc74319ec423b9c72ec232f46565: Status 404 returned error can't find the container with id 1e2c73eb051610e98052d9e632114c64452dfc74319ec423b9c72ec232f46565 Jul 15 19:22:55.301775 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:55.301741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" event={"ID":"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030","Type":"ContainerStarted","Data":"1e2c73eb051610e98052d9e632114c64452dfc74319ec423b9c72ec232f46565"} Jul 15 19:22:57.409696 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:57.409660 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:22:57.409950 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:57.409740 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:22:57.409950 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:57.409769 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:22:58.316961 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:58.316924 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" event={"ID":"ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030","Type":"ContainerStarted","Data":"3b60942e65f313eeb16e87b7631dfd259358dbae38397f9439b4769a73be6c04"} Jul 15 19:22:58.338033 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:58.337976 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" podStartSLOduration=1.758176186 podStartE2EDuration="4.337962414s" totalImagesPullingTime="2.579786228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:58.336995366 +0000 UTC m=+424.173700145" watchObservedRunningTime="2026-07-15 19:22:58.337962414 +0000 UTC m=+424.174667191" Jul 15 19:22:58.684300 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:58.684261 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:58.689811 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:58.689786 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:59.321401 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:59.321368 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:22:59.322225 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:22:59.322201 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gjpgj" Jul 15 19:23:39.063114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.063075 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvgtq"] Jul 15 19:23:39.072205 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.072173 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" Jul 15 19:23:39.074870 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.074845 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 19:23:39.076287 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.076258 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9gnpw\"" Jul 15 19:23:39.076287 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.076258 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 19:23:39.078841 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.078817 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvgtq"] Jul 15 19:23:39.158272 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.158243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpjr8\" (UniqueName: \"kubernetes.io/projected/e3a3d508-aeff-414b-9b39-e00b6bf0c3d9-kube-api-access-fpjr8\") pod \"kuadrant-operator-catalog-dvgtq\" (UID: \"e3a3d508-aeff-414b-9b39-e00b6bf0c3d9\") " pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" Jul 15 19:23:39.259383 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.259340 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpjr8\" (UniqueName: \"kubernetes.io/projected/e3a3d508-aeff-414b-9b39-e00b6bf0c3d9-kube-api-access-fpjr8\") pod \"kuadrant-operator-catalog-dvgtq\" (UID: \"e3a3d508-aeff-414b-9b39-e00b6bf0c3d9\") " pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" Jul 15 19:23:39.267906 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.267882 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpjr8\" (UniqueName: \"kubernetes.io/projected/e3a3d508-aeff-414b-9b39-e00b6bf0c3d9-kube-api-access-fpjr8\") pod \"kuadrant-operator-catalog-dvgtq\" (UID: \"e3a3d508-aeff-414b-9b39-e00b6bf0c3d9\") " pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" Jul 15 19:23:39.384802 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.384773 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" Jul 15 19:23:39.415004 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.414972 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvgtq"] Jul 15 19:23:39.525828 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.525803 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvgtq"] Jul 15 19:23:39.527785 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:23:39.527747 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a3d508_aeff_414b_9b39_e00b6bf0c3d9.slice/crio-221b8fec5c5656521be84c27b282f172be6934f08cfebb970bbc4770658a4f0f WatchSource:0}: Error finding container 221b8fec5c5656521be84c27b282f172be6934f08cfebb970bbc4770658a4f0f: Status 404 returned error can't find the container with id 221b8fec5c5656521be84c27b282f172be6934f08cfebb970bbc4770658a4f0f Jul 15 19:23:39.625340 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.625309 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c4p4r"] Jul 15 19:23:39.629809 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.629794 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" Jul 15 19:23:39.637382 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.637357 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c4p4r"] Jul 15 19:23:39.661947 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.661923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn2nk\" (UniqueName: \"kubernetes.io/projected/341502b5-1a44-4fd3-9d39-888cc00721ab-kube-api-access-gn2nk\") pod \"kuadrant-operator-catalog-c4p4r\" (UID: \"341502b5-1a44-4fd3-9d39-888cc00721ab\") " pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" Jul 15 19:23:39.762499 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.762472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn2nk\" (UniqueName: \"kubernetes.io/projected/341502b5-1a44-4fd3-9d39-888cc00721ab-kube-api-access-gn2nk\") pod \"kuadrant-operator-catalog-c4p4r\" (UID: \"341502b5-1a44-4fd3-9d39-888cc00721ab\") " pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" Jul 15 19:23:39.771260 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.771231 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn2nk\" (UniqueName: \"kubernetes.io/projected/341502b5-1a44-4fd3-9d39-888cc00721ab-kube-api-access-gn2nk\") pod \"kuadrant-operator-catalog-c4p4r\" (UID: \"341502b5-1a44-4fd3-9d39-888cc00721ab\") " pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" Jul 15 19:23:39.940247 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:39.940170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" Jul 15 19:23:40.065285 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:40.065258 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c4p4r"] Jul 15 19:23:40.067336 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:23:40.067310 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341502b5_1a44_4fd3_9d39_888cc00721ab.slice/crio-fb410ac59f5962c45915e690a0aee44ee0cf3d5e9eb665c184103146bd88a191 WatchSource:0}: Error finding container fb410ac59f5962c45915e690a0aee44ee0cf3d5e9eb665c184103146bd88a191: Status 404 returned error can't find the container with id fb410ac59f5962c45915e690a0aee44ee0cf3d5e9eb665c184103146bd88a191 Jul 15 19:23:40.503631 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:40.503592 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" event={"ID":"e3a3d508-aeff-414b-9b39-e00b6bf0c3d9","Type":"ContainerStarted","Data":"221b8fec5c5656521be84c27b282f172be6934f08cfebb970bbc4770658a4f0f"} Jul 15 19:23:40.504784 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:40.504753 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" event={"ID":"341502b5-1a44-4fd3-9d39-888cc00721ab","Type":"ContainerStarted","Data":"fb410ac59f5962c45915e690a0aee44ee0cf3d5e9eb665c184103146bd88a191"} Jul 15 19:23:42.515841 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:42.515806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" event={"ID":"341502b5-1a44-4fd3-9d39-888cc00721ab","Type":"ContainerStarted","Data":"7ccdc10a103ddfac2c2cf10da412ae1bada686f5fc38dd6e98646c0b03b49934"} Jul 15 19:23:42.517224 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:42.517197 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" event={"ID":"e3a3d508-aeff-414b-9b39-e00b6bf0c3d9","Type":"ContainerStarted","Data":"65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065"} Jul 15 19:23:42.517322 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:42.517276 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" podUID="e3a3d508-aeff-414b-9b39-e00b6bf0c3d9" containerName="registry-server" containerID="cri-o://65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065" gracePeriod=2 Jul 15 19:23:42.532289 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:42.532244 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" podStartSLOduration=1.728474077 podStartE2EDuration="3.53222925s" totalImagesPullingTime="1.803755173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:23:42.530884156 +0000 UTC m=+468.367588933" watchObservedRunningTime="2026-07-15 19:23:42.53222925 +0000 UTC m=+468.368934026" Jul 15 19:23:42.547107 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:42.547066 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" podStartSLOduration=1.204678767 podStartE2EDuration="3.547052177s" totalImagesPullingTime="2.34237341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:23:42.545451843 +0000 UTC m=+468.382156620" watchObservedRunningTime="2026-07-15 19:23:42.547052177 +0000 UTC m=+468.383756953" Jul 15 19:23:42.757986 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:42.757963 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" Jul 15 19:23:42.785341 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:42.785280 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpjr8\" (UniqueName: \"kubernetes.io/projected/e3a3d508-aeff-414b-9b39-e00b6bf0c3d9-kube-api-access-fpjr8\") pod \"e3a3d508-aeff-414b-9b39-e00b6bf0c3d9\" (UID: \"e3a3d508-aeff-414b-9b39-e00b6bf0c3d9\") " Jul 15 19:23:42.787565 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:42.787540 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3a3d508-aeff-414b-9b39-e00b6bf0c3d9-kube-api-access-fpjr8" (OuterVolumeSpecName: "kube-api-access-fpjr8") pod "e3a3d508-aeff-414b-9b39-e00b6bf0c3d9" (UID: "e3a3d508-aeff-414b-9b39-e00b6bf0c3d9"). InnerVolumeSpecName "kube-api-access-fpjr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:23:42.885928 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:42.885900 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpjr8\" (UniqueName: \"kubernetes.io/projected/e3a3d508-aeff-414b-9b39-e00b6bf0c3d9-kube-api-access-fpjr8\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:23:43.522635 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:43.522599 2562 generic.go:356] "Generic (PLEG): container finished" podID="e3a3d508-aeff-414b-9b39-e00b6bf0c3d9" containerID="65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065" exitCode=0 Jul 15 19:23:43.522635 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:43.522638 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" Jul 15 19:23:43.523127 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:43.522674 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" event={"ID":"e3a3d508-aeff-414b-9b39-e00b6bf0c3d9","Type":"ContainerDied","Data":"65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065"} Jul 15 19:23:43.523127 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:43.522719 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dvgtq" event={"ID":"e3a3d508-aeff-414b-9b39-e00b6bf0c3d9","Type":"ContainerDied","Data":"221b8fec5c5656521be84c27b282f172be6934f08cfebb970bbc4770658a4f0f"} Jul 15 19:23:43.523127 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:43.522738 2562 scope.go:117] "RemoveContainer" containerID="65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065" Jul 15 19:23:43.531181 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:43.531165 2562 scope.go:117] "RemoveContainer" containerID="65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065" Jul 15 19:23:43.531440 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:23:43.531421 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065\": container with ID starting with 65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065 not found: ID does not exist" containerID="65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065" Jul 15 19:23:43.531513 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:43.531452 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065"} err="failed to get container status \"65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065\": rpc error: code = NotFound desc = could not find container \"65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065\": container with ID starting with 65b030741a9b41abd0864d208351571a9392de277455dfbf4d2b21ff21da9065 not found: ID does not exist" Jul 15 19:23:43.542140 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:43.542114 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvgtq"] Jul 15 19:23:43.546290 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:43.546264 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dvgtq"] Jul 15 19:23:44.742187 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:44.742156 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3a3d508-aeff-414b-9b39-e00b6bf0c3d9" path="/var/lib/kubelet/pods/e3a3d508-aeff-414b-9b39-e00b6bf0c3d9/volumes" Jul 15 19:23:49.941302 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:49.941269 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" Jul 15 19:23:49.941711 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:49.941325 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" Jul 15 19:23:49.963554 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:49.963521 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" Jul 15 19:23:50.576308 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:50.576284 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-c4p4r" Jul 15 19:23:54.430591 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.430538 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb"] Jul 15 19:23:54.430955 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.430854 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3a3d508-aeff-414b-9b39-e00b6bf0c3d9" containerName="registry-server" Jul 15 19:23:54.430955 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.430866 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3a3d508-aeff-414b-9b39-e00b6bf0c3d9" containerName="registry-server" Jul 15 19:23:54.430955 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.430916 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3a3d508-aeff-414b-9b39-e00b6bf0c3d9" containerName="registry-server" Jul 15 19:23:54.433830 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.433814 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.436487 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.436469 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ds664\"" Jul 15 19:23:54.443081 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.443059 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb"] Jul 15 19:23:54.484610 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.484567 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgt6p\" (UniqueName: \"kubernetes.io/projected/e96879d9-963c-4145-9ca3-feed596fbd45-kube-api-access-kgt6p\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.484781 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.484625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.484781 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.484712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.585931 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.585907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.586026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.585986 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgt6p\" (UniqueName: \"kubernetes.io/projected/e96879d9-963c-4145-9ca3-feed596fbd45-kube-api-access-kgt6p\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.586026 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.586015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.586319 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.586298 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.586362 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.586315 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.594142 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.594119 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgt6p\" (UniqueName: \"kubernetes.io/projected/e96879d9-963c-4145-9ca3-feed596fbd45-kube-api-access-kgt6p\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.746291 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.746221 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ds664\"" Jul 15 19:23:54.754339 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.754325 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:54.882211 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:54.882183 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb"] Jul 15 19:23:54.883869 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:23:54.883846 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96879d9_963c_4145_9ca3_feed596fbd45.slice/crio-bcb1322d64ddba84cfd5e44462efa56e3c441a0fea3ab74ac492ffe67f34aa74 WatchSource:0}: Error finding container bcb1322d64ddba84cfd5e44462efa56e3c441a0fea3ab74ac492ffe67f34aa74: Status 404 returned error can't find the container with id bcb1322d64ddba84cfd5e44462efa56e3c441a0fea3ab74ac492ffe67f34aa74 Jul 15 19:23:55.037276 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.037201 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2"] Jul 15 19:23:55.040358 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.040343 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.048953 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.048932 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2"] Jul 15 19:23:55.090921 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.090887 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n4v2\" (UniqueName: \"kubernetes.io/projected/d7ef42c9-4e6f-4d38-91b6-6138510417ec-kube-api-access-7n4v2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.091052 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.090938 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.091052 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.090972 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.191869 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.191833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n4v2\" (UniqueName: \"kubernetes.io/projected/d7ef42c9-4e6f-4d38-91b6-6138510417ec-kube-api-access-7n4v2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.192031 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.191892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.192031 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.191931 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.192290 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.192270 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.192362 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.192300 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.200537 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.200515 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n4v2\" (UniqueName: \"kubernetes.io/projected/d7ef42c9-4e6f-4d38-91b6-6138510417ec-kube-api-access-7n4v2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.369491 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.369407 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:55.432778 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.432747 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k"] Jul 15 19:23:55.439512 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.439481 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.446438 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.446413 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k"] Jul 15 19:23:55.494698 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.494659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs9j2\" (UniqueName: \"kubernetes.io/projected/e1c23338-2e3f-4c9f-a770-d669cefbe419-kube-api-access-zs9j2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.494847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.494706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.494847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.494732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.501853 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.501827 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2"] Jul 15 19:23:55.503564 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:23:55.503542 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ef42c9_4e6f_4d38_91b6_6138510417ec.slice/crio-56c303d4c47533eec21b97b203e85faa219b894e249af16877c4462ff9021156 WatchSource:0}: Error finding container 56c303d4c47533eec21b97b203e85faa219b894e249af16877c4462ff9021156: Status 404 returned error can't find the container with id 56c303d4c47533eec21b97b203e85faa219b894e249af16877c4462ff9021156 Jul 15 19:23:55.578940 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.578905 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" event={"ID":"d7ef42c9-4e6f-4d38-91b6-6138510417ec","Type":"ContainerStarted","Data":"b2c1f9888c2f949a6985780f4635f11f567cac06552a7a0b22131155df4f0762"} Jul 15 19:23:55.579110 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.578967 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" event={"ID":"d7ef42c9-4e6f-4d38-91b6-6138510417ec","Type":"ContainerStarted","Data":"56c303d4c47533eec21b97b203e85faa219b894e249af16877c4462ff9021156"} Jul 15 19:23:55.580537 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.580512 2562 generic.go:356] "Generic (PLEG): container finished" podID="e96879d9-963c-4145-9ca3-feed596fbd45" containerID="79f927ce995ec986c46995183c8a361c83874fb5734af3d5cb8b6555c6c046ed" exitCode=0 Jul 15 19:23:55.580674 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.580569 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" event={"ID":"e96879d9-963c-4145-9ca3-feed596fbd45","Type":"ContainerDied","Data":"79f927ce995ec986c46995183c8a361c83874fb5734af3d5cb8b6555c6c046ed"} Jul 15 19:23:55.580674 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.580633 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" event={"ID":"e96879d9-963c-4145-9ca3-feed596fbd45","Type":"ContainerStarted","Data":"bcb1322d64ddba84cfd5e44462efa56e3c441a0fea3ab74ac492ffe67f34aa74"} Jul 15 19:23:55.595433 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.595176 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs9j2\" (UniqueName: \"kubernetes.io/projected/e1c23338-2e3f-4c9f-a770-d669cefbe419-kube-api-access-zs9j2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.595433 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.595218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.595433 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.595255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.595861 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.595816 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.596053 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.596019 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.603345 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.603327 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs9j2\" (UniqueName: \"kubernetes.io/projected/e1c23338-2e3f-4c9f-a770-d669cefbe419-kube-api-access-zs9j2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.751218 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.751182 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:55.831841 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.831807 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2"] Jul 15 19:23:55.836713 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.836685 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:55.843050 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.842986 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2"] Jul 15 19:23:55.885564 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.885543 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k"] Jul 15 19:23:55.887282 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:23:55.887258 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c23338_2e3f_4c9f_a770_d669cefbe419.slice/crio-c90515c3ae7b1255891fc7e90c79853afebf808a3379db596e1a4f23a9eaa164 WatchSource:0}: Error finding container c90515c3ae7b1255891fc7e90c79853afebf808a3379db596e1a4f23a9eaa164: Status 404 returned error can't find the container with id c90515c3ae7b1255891fc7e90c79853afebf808a3379db596e1a4f23a9eaa164 Jul 15 19:23:55.897444 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.897418 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:55.897561 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.897452 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:55.897561 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.897512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n555\" (UniqueName: \"kubernetes.io/projected/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-kube-api-access-4n555\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:55.997890 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.997863 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:55.998021 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.997906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:55.998021 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.997966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n555\" (UniqueName: \"kubernetes.io/projected/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-kube-api-access-4n555\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:55.998230 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.998206 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:55.998294 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:55.998257 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:56.007925 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.007875 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n555\" (UniqueName: \"kubernetes.io/projected/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-kube-api-access-4n555\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:56.148003 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.147954 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:23:56.275138 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.275109 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2"] Jul 15 19:23:56.276542 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:23:56.276520 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600f7296_13ac_4f3e_a8a8_4a8a6baf11a1.slice/crio-38747ee5c3210763d9cde184d5f75e7566a46e0058efe2b44c6d4f2da584b8c2 WatchSource:0}: Error finding container 38747ee5c3210763d9cde184d5f75e7566a46e0058efe2b44c6d4f2da584b8c2: Status 404 returned error can't find the container with id 38747ee5c3210763d9cde184d5f75e7566a46e0058efe2b44c6d4f2da584b8c2 Jul 15 19:23:56.587566 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.587528 2562 generic.go:356] "Generic (PLEG): container finished" podID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerID="b2c1f9888c2f949a6985780f4635f11f567cac06552a7a0b22131155df4f0762" exitCode=0 Jul 15 19:23:56.587967 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.587618 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" event={"ID":"d7ef42c9-4e6f-4d38-91b6-6138510417ec","Type":"ContainerDied","Data":"b2c1f9888c2f949a6985780f4635f11f567cac06552a7a0b22131155df4f0762"} Jul 15 19:23:56.589668 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.589634 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" event={"ID":"e96879d9-963c-4145-9ca3-feed596fbd45","Type":"ContainerStarted","Data":"c900c944fc8e31c55a656e96e0d9903e429ad2b0608c17c517a8e36b60da659c"} Jul 15 19:23:56.591234 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.591212 2562 generic.go:356] "Generic (PLEG): container finished" podID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerID="1367bb7d2f89dfc2eb648ea076207b687dba7e82b9a87b5f8e5753ab91169fe2" exitCode=0 Jul 15 19:23:56.591339 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.591296 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" event={"ID":"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1","Type":"ContainerDied","Data":"1367bb7d2f89dfc2eb648ea076207b687dba7e82b9a87b5f8e5753ab91169fe2"} Jul 15 19:23:56.591339 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.591331 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" event={"ID":"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1","Type":"ContainerStarted","Data":"38747ee5c3210763d9cde184d5f75e7566a46e0058efe2b44c6d4f2da584b8c2"} Jul 15 19:23:56.592874 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.592852 2562 generic.go:356] "Generic (PLEG): container finished" podID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerID="280919564d676e880e6b34759f86b67ebd2fee6395b2d1208fc2a797e86bb22c" exitCode=0 Jul 15 19:23:56.592991 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.592884 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" event={"ID":"e1c23338-2e3f-4c9f-a770-d669cefbe419","Type":"ContainerDied","Data":"280919564d676e880e6b34759f86b67ebd2fee6395b2d1208fc2a797e86bb22c"} Jul 15 19:23:56.592991 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:56.592908 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" event={"ID":"e1c23338-2e3f-4c9f-a770-d669cefbe419","Type":"ContainerStarted","Data":"c90515c3ae7b1255891fc7e90c79853afebf808a3379db596e1a4f23a9eaa164"} Jul 15 19:23:57.599680 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:57.599609 2562 generic.go:356] "Generic (PLEG): container finished" podID="e96879d9-963c-4145-9ca3-feed596fbd45" containerID="c900c944fc8e31c55a656e96e0d9903e429ad2b0608c17c517a8e36b60da659c" exitCode=0 Jul 15 19:23:57.599680 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:57.599640 2562 generic.go:356] "Generic (PLEG): container finished" podID="e96879d9-963c-4145-9ca3-feed596fbd45" containerID="76b72557653c3f9930ee0efbb67462a5a7054b7c7c7d6e5dee8e53b8dff0c0f5" exitCode=0 Jul 15 19:23:57.600092 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:57.599718 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" event={"ID":"e96879d9-963c-4145-9ca3-feed596fbd45","Type":"ContainerDied","Data":"c900c944fc8e31c55a656e96e0d9903e429ad2b0608c17c517a8e36b60da659c"} Jul 15 19:23:57.600092 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:57.599778 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" event={"ID":"e96879d9-963c-4145-9ca3-feed596fbd45","Type":"ContainerDied","Data":"76b72557653c3f9930ee0efbb67462a5a7054b7c7c7d6e5dee8e53b8dff0c0f5"} Jul 15 19:23:57.601722 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:57.601697 2562 generic.go:356] "Generic (PLEG): container finished" podID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerID="15cfbaebb51b06eec6920b78d3f839ce7864a07955fe471cf8cbdc72adb92183" exitCode=0 Jul 15 19:23:57.601842 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:57.601821 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" event={"ID":"e1c23338-2e3f-4c9f-a770-d669cefbe419","Type":"ContainerDied","Data":"15cfbaebb51b06eec6920b78d3f839ce7864a07955fe471cf8cbdc72adb92183"} Jul 15 19:23:57.603778 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:57.603757 2562 generic.go:356] "Generic (PLEG): container finished" podID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerID="90ec377667f9f6aceeae510c1a2fd9ed19b29ba556756f9ef749af7817db2785" exitCode=0 Jul 15 19:23:57.603865 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:57.603796 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" event={"ID":"d7ef42c9-4e6f-4d38-91b6-6138510417ec","Type":"ContainerDied","Data":"90ec377667f9f6aceeae510c1a2fd9ed19b29ba556756f9ef749af7817db2785"} Jul 15 19:23:58.610652 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.610618 2562 generic.go:356] "Generic (PLEG): container finished" podID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerID="ffedb0a13aa7dd4e0923cad542bf403532138fc1b1e63c8762310990f5ffd8e1" exitCode=0 Jul 15 19:23:58.611066 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.610706 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" event={"ID":"e1c23338-2e3f-4c9f-a770-d669cefbe419","Type":"ContainerDied","Data":"ffedb0a13aa7dd4e0923cad542bf403532138fc1b1e63c8762310990f5ffd8e1"} Jul 15 19:23:58.612773 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.612754 2562 generic.go:356] "Generic (PLEG): container finished" podID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerID="d8ca537d97ccd8b077ab8d62dc3e1ad0065d218482fb5119736a5dfd25dcb2ca" exitCode=0 Jul 15 19:23:58.612847 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.612831 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" event={"ID":"d7ef42c9-4e6f-4d38-91b6-6138510417ec","Type":"ContainerDied","Data":"d8ca537d97ccd8b077ab8d62dc3e1ad0065d218482fb5119736a5dfd25dcb2ca"} Jul 15 19:23:58.614497 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.614473 2562 generic.go:356] "Generic (PLEG): container finished" podID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerID="212411202c7af5d08c2ee754ca1bcffa5aec0d1d4ce41d94927256466655a624" exitCode=0 Jul 15 19:23:58.614623 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.614562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" event={"ID":"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1","Type":"ContainerDied","Data":"212411202c7af5d08c2ee754ca1bcffa5aec0d1d4ce41d94927256466655a624"} Jul 15 19:23:58.802425 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.802401 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:58.921187 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.921158 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-util\") pod \"e96879d9-963c-4145-9ca3-feed596fbd45\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " Jul 15 19:23:58.921341 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.921199 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgt6p\" (UniqueName: \"kubernetes.io/projected/e96879d9-963c-4145-9ca3-feed596fbd45-kube-api-access-kgt6p\") pod \"e96879d9-963c-4145-9ca3-feed596fbd45\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " Jul 15 19:23:58.921341 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.921287 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-bundle\") pod \"e96879d9-963c-4145-9ca3-feed596fbd45\" (UID: \"e96879d9-963c-4145-9ca3-feed596fbd45\") " Jul 15 19:23:58.921918 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.921893 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-bundle" (OuterVolumeSpecName: "bundle") pod "e96879d9-963c-4145-9ca3-feed596fbd45" (UID: "e96879d9-963c-4145-9ca3-feed596fbd45"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:23:58.923516 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.923457 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e96879d9-963c-4145-9ca3-feed596fbd45-kube-api-access-kgt6p" (OuterVolumeSpecName: "kube-api-access-kgt6p") pod "e96879d9-963c-4145-9ca3-feed596fbd45" (UID: "e96879d9-963c-4145-9ca3-feed596fbd45"). InnerVolumeSpecName "kube-api-access-kgt6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:23:58.927035 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:58.927015 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-util" (OuterVolumeSpecName: "util") pod "e96879d9-963c-4145-9ca3-feed596fbd45" (UID: "e96879d9-963c-4145-9ca3-feed596fbd45"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:23:59.021782 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.021747 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgt6p\" (UniqueName: \"kubernetes.io/projected/e96879d9-963c-4145-9ca3-feed596fbd45-kube-api-access-kgt6p\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:23:59.021782 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.021779 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:23:59.021782 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.021788 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e96879d9-963c-4145-9ca3-feed596fbd45-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:23:59.621908 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.621871 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" event={"ID":"e96879d9-963c-4145-9ca3-feed596fbd45","Type":"ContainerDied","Data":"bcb1322d64ddba84cfd5e44462efa56e3c441a0fea3ab74ac492ffe67f34aa74"} Jul 15 19:23:59.621908 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.621891 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb" Jul 15 19:23:59.622361 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.621923 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcb1322d64ddba84cfd5e44462efa56e3c441a0fea3ab74ac492ffe67f34aa74" Jul 15 19:23:59.623926 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.623899 2562 generic.go:356] "Generic (PLEG): container finished" podID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerID="ae0699ad56e3cfe1807689a4742141f890804ef56e70ab58564368ea6b6c4bc6" exitCode=0 Jul 15 19:23:59.624067 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.623975 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" event={"ID":"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1","Type":"ContainerDied","Data":"ae0699ad56e3cfe1807689a4742141f890804ef56e70ab58564368ea6b6c4bc6"} Jul 15 19:23:59.748419 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.748397 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:23:59.778235 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.778203 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:23:59.828278 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.828247 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n4v2\" (UniqueName: \"kubernetes.io/projected/d7ef42c9-4e6f-4d38-91b6-6138510417ec-kube-api-access-7n4v2\") pod \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " Jul 15 19:23:59.828459 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.828292 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs9j2\" (UniqueName: \"kubernetes.io/projected/e1c23338-2e3f-4c9f-a770-d669cefbe419-kube-api-access-zs9j2\") pod \"e1c23338-2e3f-4c9f-a770-d669cefbe419\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " Jul 15 19:23:59.828459 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.828323 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-util\") pod \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " Jul 15 19:23:59.828459 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.828350 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-bundle\") pod \"e1c23338-2e3f-4c9f-a770-d669cefbe419\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " Jul 15 19:23:59.828459 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.828366 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-util\") pod \"e1c23338-2e3f-4c9f-a770-d669cefbe419\" (UID: \"e1c23338-2e3f-4c9f-a770-d669cefbe419\") " Jul 15 19:23:59.828459 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.828424 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-bundle\") pod \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\" (UID: \"d7ef42c9-4e6f-4d38-91b6-6138510417ec\") " Jul 15 19:23:59.829075 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.829042 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-bundle" (OuterVolumeSpecName: "bundle") pod "e1c23338-2e3f-4c9f-a770-d669cefbe419" (UID: "e1c23338-2e3f-4c9f-a770-d669cefbe419"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:23:59.829075 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.829048 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-bundle" (OuterVolumeSpecName: "bundle") pod "d7ef42c9-4e6f-4d38-91b6-6138510417ec" (UID: "d7ef42c9-4e6f-4d38-91b6-6138510417ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:23:59.830729 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.830698 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ef42c9-4e6f-4d38-91b6-6138510417ec-kube-api-access-7n4v2" (OuterVolumeSpecName: "kube-api-access-7n4v2") pod "d7ef42c9-4e6f-4d38-91b6-6138510417ec" (UID: "d7ef42c9-4e6f-4d38-91b6-6138510417ec"). InnerVolumeSpecName "kube-api-access-7n4v2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:23:59.830729 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.830714 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c23338-2e3f-4c9f-a770-d669cefbe419-kube-api-access-zs9j2" (OuterVolumeSpecName: "kube-api-access-zs9j2") pod "e1c23338-2e3f-4c9f-a770-d669cefbe419" (UID: "e1c23338-2e3f-4c9f-a770-d669cefbe419"). InnerVolumeSpecName "kube-api-access-zs9j2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:23:59.836262 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.836232 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-util" (OuterVolumeSpecName: "util") pod "e1c23338-2e3f-4c9f-a770-d669cefbe419" (UID: "e1c23338-2e3f-4c9f-a770-d669cefbe419"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:23:59.836888 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.836871 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-util" (OuterVolumeSpecName: "util") pod "d7ef42c9-4e6f-4d38-91b6-6138510417ec" (UID: "d7ef42c9-4e6f-4d38-91b6-6138510417ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:23:59.929720 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.929681 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:23:59.929720 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.929715 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n4v2\" (UniqueName: \"kubernetes.io/projected/d7ef42c9-4e6f-4d38-91b6-6138510417ec-kube-api-access-7n4v2\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:23:59.929720 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.929724 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs9j2\" (UniqueName: \"kubernetes.io/projected/e1c23338-2e3f-4c9f-a770-d669cefbe419-kube-api-access-zs9j2\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:23:59.929947 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.929732 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7ef42c9-4e6f-4d38-91b6-6138510417ec-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:23:59.929947 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.929742 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:23:59.929947 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:23:59.929749 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1c23338-2e3f-4c9f-a770-d669cefbe419-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:24:00.630443 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.630416 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" Jul 15 19:24:00.630918 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.630455 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k" event={"ID":"e1c23338-2e3f-4c9f-a770-d669cefbe419","Type":"ContainerDied","Data":"c90515c3ae7b1255891fc7e90c79853afebf808a3379db596e1a4f23a9eaa164"} Jul 15 19:24:00.630918 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.630498 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c90515c3ae7b1255891fc7e90c79853afebf808a3379db596e1a4f23a9eaa164" Jul 15 19:24:00.638491 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.638461 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" Jul 15 19:24:00.638491 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.638461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2" event={"ID":"d7ef42c9-4e6f-4d38-91b6-6138510417ec","Type":"ContainerDied","Data":"56c303d4c47533eec21b97b203e85faa219b894e249af16877c4462ff9021156"} Jul 15 19:24:00.638694 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.638558 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56c303d4c47533eec21b97b203e85faa219b894e249af16877c4462ff9021156" Jul 15 19:24:00.762298 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.762274 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:24:00.838074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.838043 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-util\") pod \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " Jul 15 19:24:00.838255 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.838137 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n555\" (UniqueName: \"kubernetes.io/projected/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-kube-api-access-4n555\") pod \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " Jul 15 19:24:00.838255 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.838163 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-bundle\") pod \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\" (UID: \"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1\") " Jul 15 19:24:00.838693 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.838665 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-bundle" (OuterVolumeSpecName: "bundle") pod "600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" (UID: "600f7296-13ac-4f3e-a8a8-4a8a6baf11a1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:24:00.840402 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.840374 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-kube-api-access-4n555" (OuterVolumeSpecName: "kube-api-access-4n555") pod "600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" (UID: "600f7296-13ac-4f3e-a8a8-4a8a6baf11a1"). InnerVolumeSpecName "kube-api-access-4n555". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:24:00.843350 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.843327 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-util" (OuterVolumeSpecName: "util") pod "600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" (UID: "600f7296-13ac-4f3e-a8a8-4a8a6baf11a1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:24:00.938948 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.938912 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-util\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:24:00.938948 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.938939 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n555\" (UniqueName: \"kubernetes.io/projected/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-kube-api-access-4n555\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:24:00.938948 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:00.938950 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/600f7296-13ac-4f3e-a8a8-4a8a6baf11a1-bundle\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:24:01.645151 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:01.645121 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" Jul 15 19:24:01.645536 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:01.645124 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2" event={"ID":"600f7296-13ac-4f3e-a8a8-4a8a6baf11a1","Type":"ContainerDied","Data":"38747ee5c3210763d9cde184d5f75e7566a46e0058efe2b44c6d4f2da584b8c2"} Jul 15 19:24:01.645536 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:01.645244 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38747ee5c3210763d9cde184d5f75e7566a46e0058efe2b44c6d4f2da584b8c2" Jul 15 19:24:11.137123 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137083 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq"] Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137396 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerName="extract" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137413 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerName="extract" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137429 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e96879d9-963c-4145-9ca3-feed596fbd45" containerName="extract" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137437 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e96879d9-963c-4145-9ca3-feed596fbd45" containerName="extract" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137446 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerName="util" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137452 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerName="util" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137458 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e96879d9-963c-4145-9ca3-feed596fbd45" containerName="pull" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137462 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e96879d9-963c-4145-9ca3-feed596fbd45" containerName="pull" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137470 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerName="extract" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137475 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerName="extract" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137481 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e96879d9-963c-4145-9ca3-feed596fbd45" containerName="util" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137487 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e96879d9-963c-4145-9ca3-feed596fbd45" containerName="util" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137492 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerName="util" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137497 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerName="util" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137502 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerName="pull" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137506 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerName="pull" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137511 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerName="pull" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137516 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerName="pull" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137523 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerName="util" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137527 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerName="util" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137532 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerName="pull" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137537 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerName="pull" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137543 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerName="extract" Jul 15 19:24:11.137547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137547 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerName="extract" Jul 15 19:24:11.138521 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137632 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e96879d9-963c-4145-9ca3-feed596fbd45" containerName="extract" Jul 15 19:24:11.138521 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137639 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ef42c9-4e6f-4d38-91b6-6138510417ec" containerName="extract" Jul 15 19:24:11.138521 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137647 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1c23338-2e3f-4c9f-a770-d669cefbe419" containerName="extract" Jul 15 19:24:11.138521 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.137655 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="600f7296-13ac-4f3e-a8a8-4a8a6baf11a1" containerName="extract" Jul 15 19:24:11.140902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.140884 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:11.143456 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.143438 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-288zc\"" Jul 15 19:24:11.154627 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.154568 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq"] Jul 15 19:24:11.222488 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.222450 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5c6add8-f1b7-4cd1-8054-db65f30da641-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" (UID: \"d5c6add8-f1b7-4cd1-8054-db65f30da641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:11.222640 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.222504 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2m5g\" (UniqueName: \"kubernetes.io/projected/d5c6add8-f1b7-4cd1-8054-db65f30da641-kube-api-access-w2m5g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" (UID: \"d5c6add8-f1b7-4cd1-8054-db65f30da641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:11.322968 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.322925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2m5g\" (UniqueName: \"kubernetes.io/projected/d5c6add8-f1b7-4cd1-8054-db65f30da641-kube-api-access-w2m5g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" (UID: \"d5c6add8-f1b7-4cd1-8054-db65f30da641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:11.323149 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.323033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5c6add8-f1b7-4cd1-8054-db65f30da641-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" (UID: \"d5c6add8-f1b7-4cd1-8054-db65f30da641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:11.323367 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.323350 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5c6add8-f1b7-4cd1-8054-db65f30da641-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" (UID: \"d5c6add8-f1b7-4cd1-8054-db65f30da641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:11.334355 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.334322 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2m5g\" (UniqueName: \"kubernetes.io/projected/d5c6add8-f1b7-4cd1-8054-db65f30da641-kube-api-access-w2m5g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" (UID: \"d5c6add8-f1b7-4cd1-8054-db65f30da641\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:11.452141 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.452054 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:11.582772 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.582742 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq"] Jul 15 19:24:11.584551 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:24:11.584524 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c6add8_f1b7_4cd1_8054_db65f30da641.slice/crio-8fe8c701a34d3e2083c08cd5c59322d2cca60f8f54c8887989dcf27c8b86d941 WatchSource:0}: Error finding container 8fe8c701a34d3e2083c08cd5c59322d2cca60f8f54c8887989dcf27c8b86d941: Status 404 returned error can't find the container with id 8fe8c701a34d3e2083c08cd5c59322d2cca60f8f54c8887989dcf27c8b86d941 Jul 15 19:24:11.690652 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:11.690608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" event={"ID":"d5c6add8-f1b7-4cd1-8054-db65f30da641","Type":"ContainerStarted","Data":"8fe8c701a34d3e2083c08cd5c59322d2cca60f8f54c8887989dcf27c8b86d941"} Jul 15 19:24:14.341729 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.341678 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r"] Jul 15 19:24:14.345006 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.344988 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" Jul 15 19:24:14.347981 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.347958 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 15 19:24:14.348066 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.347990 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-skw7k\"" Jul 15 19:24:14.355162 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.355142 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r"] Jul 15 19:24:14.449250 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.449214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf9q2\" (UniqueName: \"kubernetes.io/projected/7c822e55-0ece-4872-ab2d-185dde5b050d-kube-api-access-wf9q2\") pod \"dns-operator-controller-manager-864dd78c4c-ps79r\" (UID: \"7c822e55-0ece-4872-ab2d-185dde5b050d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" Jul 15 19:24:14.550006 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.549961 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf9q2\" (UniqueName: \"kubernetes.io/projected/7c822e55-0ece-4872-ab2d-185dde5b050d-kube-api-access-wf9q2\") pod \"dns-operator-controller-manager-864dd78c4c-ps79r\" (UID: \"7c822e55-0ece-4872-ab2d-185dde5b050d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" Jul 15 19:24:14.564088 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.564058 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf9q2\" (UniqueName: \"kubernetes.io/projected/7c822e55-0ece-4872-ab2d-185dde5b050d-kube-api-access-wf9q2\") pod \"dns-operator-controller-manager-864dd78c4c-ps79r\" (UID: \"7c822e55-0ece-4872-ab2d-185dde5b050d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" Jul 15 19:24:14.655911 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.655870 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" Jul 15 19:24:14.827005 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:14.826836 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r"] Jul 15 19:24:14.829998 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:24:14.829965 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c822e55_0ece_4872_ab2d_185dde5b050d.slice/crio-147f74d0d1d8e701674e29f4a5eb1dd74d4cdec67a8cc1f36a298cc645ba6b67 WatchSource:0}: Error finding container 147f74d0d1d8e701674e29f4a5eb1dd74d4cdec67a8cc1f36a298cc645ba6b67: Status 404 returned error can't find the container with id 147f74d0d1d8e701674e29f4a5eb1dd74d4cdec67a8cc1f36a298cc645ba6b67 Jul 15 19:24:15.713905 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:15.713859 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" event={"ID":"7c822e55-0ece-4872-ab2d-185dde5b050d","Type":"ContainerStarted","Data":"147f74d0d1d8e701674e29f4a5eb1dd74d4cdec67a8cc1f36a298cc645ba6b67"} Jul 15 19:24:17.597868 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.597781 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j"] Jul 15 19:24:17.601478 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.601451 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:17.604942 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.604921 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-x2j22\"" Jul 15 19:24:17.645486 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.645453 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j"] Jul 15 19:24:17.681342 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.681306 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzlqs\" (UniqueName: \"kubernetes.io/projected/7eeac26d-05e2-4c5f-bc7f-096b01077081-kube-api-access-rzlqs\") pod \"limitador-operator-controller-manager-859d7b78bd-f858j\" (UID: \"7eeac26d-05e2-4c5f-bc7f-096b01077081\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:17.727699 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.727664 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" event={"ID":"d5c6add8-f1b7-4cd1-8054-db65f30da641","Type":"ContainerStarted","Data":"d84da15af49d559196eea25629ee104d55094ea1ea5125c8e76d059d41840ef1"} Jul 15 19:24:17.727890 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.727804 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:17.782289 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.782244 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzlqs\" (UniqueName: \"kubernetes.io/projected/7eeac26d-05e2-4c5f-bc7f-096b01077081-kube-api-access-rzlqs\") pod \"limitador-operator-controller-manager-859d7b78bd-f858j\" (UID: \"7eeac26d-05e2-4c5f-bc7f-096b01077081\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:17.803090 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.803054 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzlqs\" (UniqueName: \"kubernetes.io/projected/7eeac26d-05e2-4c5f-bc7f-096b01077081-kube-api-access-rzlqs\") pod \"limitador-operator-controller-manager-859d7b78bd-f858j\" (UID: \"7eeac26d-05e2-4c5f-bc7f-096b01077081\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:17.807177 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.807092 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" podStartSLOduration=1.6049358630000001 podStartE2EDuration="6.807072517s" totalImagesPullingTime="5.202136654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:24:17.805513274 +0000 UTC m=+503.642218052" watchObservedRunningTime="2026-07-15 19:24:17.807072517 +0000 UTC m=+503.643777294" Jul 15 19:24:17.913940 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:17.913901 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:18.107622 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:18.107594 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j"] Jul 15 19:24:18.108818 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:24:18.108792 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eeac26d_05e2_4c5f_bc7f_096b01077081.slice/crio-d2416bf129a5a674236e7b0f2b7aaeb9ac760ff9be0ec4c379f098d973e6c624 WatchSource:0}: Error finding container d2416bf129a5a674236e7b0f2b7aaeb9ac760ff9be0ec4c379f098d973e6c624: Status 404 returned error can't find the container with id d2416bf129a5a674236e7b0f2b7aaeb9ac760ff9be0ec4c379f098d973e6c624 Jul 15 19:24:18.748671 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:18.748643 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" Jul 15 19:24:18.749135 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:18.748704 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" event={"ID":"7eeac26d-05e2-4c5f-bc7f-096b01077081","Type":"ContainerStarted","Data":"d2416bf129a5a674236e7b0f2b7aaeb9ac760ff9be0ec4c379f098d973e6c624"} Jul 15 19:24:18.749135 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:18.748735 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" event={"ID":"7c822e55-0ece-4872-ab2d-185dde5b050d","Type":"ContainerStarted","Data":"afd2222e1466e9c8b79a8c1dd3f4bd415717188c4593f736973a5461794a737b"} Jul 15 19:24:18.765558 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:18.765497 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" podStartSLOduration=1.578208875 podStartE2EDuration="4.765474677s" totalImagesPullingTime="3.187265802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:24:18.763688118 +0000 UTC m=+504.600392905" watchObservedRunningTime="2026-07-15 19:24:18.765474677 +0000 UTC m=+504.602179455" Jul 15 19:24:19.746042 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:19.745999 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" event={"ID":"7eeac26d-05e2-4c5f-bc7f-096b01077081","Type":"ContainerStarted","Data":"42399c182e6aef96571f27b1d9f37c6ebce3afa4ecf28fa0d38c63152d171692"} Jul 15 19:24:19.746201 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:19.746091 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:19.765925 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:19.765870 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" podStartSLOduration=1.296655214 podStartE2EDuration="2.765852634s" totalImagesPullingTime="1.46919742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:24:19.763834654 +0000 UTC m=+505.600539431" watchObservedRunningTime="2026-07-15 19:24:19.765852634 +0000 UTC m=+505.602557410" Jul 15 19:24:20.189346 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.189311 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx"] Jul 15 19:24:20.192687 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.192660 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.195396 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.195364 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ds664\"" Jul 15 19:24:20.195536 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.195364 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 15 19:24:20.195536 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.195370 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 15 19:24:20.201783 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.201760 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx"] Jul 15 19:24:20.303143 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.303107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d43e74ae-4bec-44ec-a141-889949d3016b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zx6jx\" (UID: \"d43e74ae-4bec-44ec-a141-889949d3016b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.303316 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.303152 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d43e74ae-4bec-44ec-a141-889949d3016b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zx6jx\" (UID: \"d43e74ae-4bec-44ec-a141-889949d3016b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.303316 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.303197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfvmb\" (UniqueName: \"kubernetes.io/projected/d43e74ae-4bec-44ec-a141-889949d3016b-kube-api-access-mfvmb\") pod \"kuadrant-console-plugin-66b5774985-zx6jx\" (UID: \"d43e74ae-4bec-44ec-a141-889949d3016b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.403858 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.403814 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfvmb\" (UniqueName: \"kubernetes.io/projected/d43e74ae-4bec-44ec-a141-889949d3016b-kube-api-access-mfvmb\") pod \"kuadrant-console-plugin-66b5774985-zx6jx\" (UID: \"d43e74ae-4bec-44ec-a141-889949d3016b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.404057 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.403883 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d43e74ae-4bec-44ec-a141-889949d3016b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zx6jx\" (UID: \"d43e74ae-4bec-44ec-a141-889949d3016b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.404057 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.403915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d43e74ae-4bec-44ec-a141-889949d3016b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zx6jx\" (UID: \"d43e74ae-4bec-44ec-a141-889949d3016b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.404500 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.404477 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d43e74ae-4bec-44ec-a141-889949d3016b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zx6jx\" (UID: \"d43e74ae-4bec-44ec-a141-889949d3016b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.406411 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.406387 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d43e74ae-4bec-44ec-a141-889949d3016b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zx6jx\" (UID: \"d43e74ae-4bec-44ec-a141-889949d3016b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.413409 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.413388 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfvmb\" (UniqueName: \"kubernetes.io/projected/d43e74ae-4bec-44ec-a141-889949d3016b-kube-api-access-mfvmb\") pod \"kuadrant-console-plugin-66b5774985-zx6jx\" (UID: \"d43e74ae-4bec-44ec-a141-889949d3016b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.502565 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.502476 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" Jul 15 19:24:20.635152 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.635124 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx"] Jul 15 19:24:20.636980 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:24:20.636954 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd43e74ae_4bec_44ec_a141_889949d3016b.slice/crio-fa89e9579bfb9e1ebeef3b87dd3d6ccb652ea66b1296e6586b4167cb02f38e29 WatchSource:0}: Error finding container fa89e9579bfb9e1ebeef3b87dd3d6ccb652ea66b1296e6586b4167cb02f38e29: Status 404 returned error can't find the container with id fa89e9579bfb9e1ebeef3b87dd3d6ccb652ea66b1296e6586b4167cb02f38e29 Jul 15 19:24:20.751347 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:20.751309 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" event={"ID":"d43e74ae-4bec-44ec-a141-889949d3016b","Type":"ContainerStarted","Data":"fa89e9579bfb9e1ebeef3b87dd3d6ccb652ea66b1296e6586b4167cb02f38e29"} Jul 15 19:24:28.743692 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:28.743658 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:30.547829 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.547793 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq"] Jul 15 19:24:30.548280 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.548005 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" containerName="manager" containerID="cri-o://d84da15af49d559196eea25629ee104d55094ea1ea5125c8e76d059d41840ef1" gracePeriod=2 Jul 15 19:24:30.552803 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.552775 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq"] Jul 15 19:24:30.581942 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.581287 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz"] Jul 15 19:24:30.582282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.582253 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" containerName="manager" Jul 15 19:24:30.582282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.582282 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" containerName="manager" Jul 15 19:24:30.582456 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.582400 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" containerName="manager" Jul 15 19:24:30.586179 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.586154 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:30.589480 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:30.588953 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:30.591665 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.591641 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j"] Jul 15 19:24:30.592341 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.592313 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" containerName="manager" containerID="cri-o://42399c182e6aef96571f27b1d9f37c6ebce3afa4ecf28fa0d38c63152d171692" gracePeriod=2 Jul 15 19:24:30.593907 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.593850 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:30.602555 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.601351 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j"] Jul 15 19:24:30.604538 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.604512 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz"] Jul 15 19:24:30.617785 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.617758 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc"] Jul 15 19:24:30.618106 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.618090 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" containerName="manager" Jul 15 19:24:30.618200 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.618107 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" containerName="manager" Jul 15 19:24:30.618200 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.618188 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" containerName="manager" Jul 15 19:24:30.621547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.621524 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" Jul 15 19:24:30.623626 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:30.623566 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f858j\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:30.626221 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:30.626192 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:30.628796 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:30.628768 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:30.631551 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:30.631389 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f858j\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:30.639621 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.639565 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc"] Jul 15 19:24:30.692596 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.692540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zx5z\" (UniqueName: \"kubernetes.io/projected/28d38c22-7196-4aa5-b2b4-91347feae83c-kube-api-access-4zx5z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dgwdz\" (UID: \"28d38c22-7196-4aa5-b2b4-91347feae83c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:30.692746 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.692632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28d38c22-7196-4aa5-b2b4-91347feae83c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dgwdz\" (UID: \"28d38c22-7196-4aa5-b2b4-91347feae83c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:30.753823 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.753788 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ps79r" Jul 15 19:24:30.794399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.793497 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28d38c22-7196-4aa5-b2b4-91347feae83c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dgwdz\" (UID: \"28d38c22-7196-4aa5-b2b4-91347feae83c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:30.794399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.793605 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px77p\" (UniqueName: \"kubernetes.io/projected/0c14da53-96cd-494a-a713-cfac1aab77b5-kube-api-access-px77p\") pod \"limitador-operator-controller-manager-859d7b78bd-wbrgc\" (UID: \"0c14da53-96cd-494a-a713-cfac1aab77b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" Jul 15 19:24:30.794399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.793662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zx5z\" (UniqueName: \"kubernetes.io/projected/28d38c22-7196-4aa5-b2b4-91347feae83c-kube-api-access-4zx5z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dgwdz\" (UID: \"28d38c22-7196-4aa5-b2b4-91347feae83c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:30.794399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.794330 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28d38c22-7196-4aa5-b2b4-91347feae83c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dgwdz\" (UID: \"28d38c22-7196-4aa5-b2b4-91347feae83c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:30.803665 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.803560 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zx5z\" (UniqueName: \"kubernetes.io/projected/28d38c22-7196-4aa5-b2b4-91347feae83c-kube-api-access-4zx5z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-dgwdz\" (UID: \"28d38c22-7196-4aa5-b2b4-91347feae83c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:30.810913 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.810879 2562 generic.go:356] "Generic (PLEG): container finished" podID="d5c6add8-f1b7-4cd1-8054-db65f30da641" containerID="d84da15af49d559196eea25629ee104d55094ea1ea5125c8e76d059d41840ef1" exitCode=0 Jul 15 19:24:30.813145 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.813048 2562 generic.go:356] "Generic (PLEG): container finished" podID="7eeac26d-05e2-4c5f-bc7f-096b01077081" containerID="42399c182e6aef96571f27b1d9f37c6ebce3afa4ecf28fa0d38c63152d171692" exitCode=0 Jul 15 19:24:30.894524 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.894489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px77p\" (UniqueName: \"kubernetes.io/projected/0c14da53-96cd-494a-a713-cfac1aab77b5-kube-api-access-px77p\") pod \"limitador-operator-controller-manager-859d7b78bd-wbrgc\" (UID: \"0c14da53-96cd-494a-a713-cfac1aab77b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" Jul 15 19:24:30.906422 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.906397 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px77p\" (UniqueName: \"kubernetes.io/projected/0c14da53-96cd-494a-a713-cfac1aab77b5-kube-api-access-px77p\") pod \"limitador-operator-controller-manager-859d7b78bd-wbrgc\" (UID: \"0c14da53-96cd-494a-a713-cfac1aab77b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" Jul 15 19:24:30.989369 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.989331 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:30.997718 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:30.997690 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" Jul 15 19:24:44.490282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.490246 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:44.492868 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:44.492811 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f858j\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:44.493716 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.493696 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:44.496721 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:44.496693 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f858j\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:44.498731 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:44.498705 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:44.523737 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.523700 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzlqs\" (UniqueName: \"kubernetes.io/projected/7eeac26d-05e2-4c5f-bc7f-096b01077081-kube-api-access-rzlqs\") pod \"7eeac26d-05e2-4c5f-bc7f-096b01077081\" (UID: \"7eeac26d-05e2-4c5f-bc7f-096b01077081\") " Jul 15 19:24:44.523893 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.523777 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5c6add8-f1b7-4cd1-8054-db65f30da641-extensions-socket-volume\") pod \"d5c6add8-f1b7-4cd1-8054-db65f30da641\" (UID: \"d5c6add8-f1b7-4cd1-8054-db65f30da641\") " Jul 15 19:24:44.523893 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.523834 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2m5g\" (UniqueName: \"kubernetes.io/projected/d5c6add8-f1b7-4cd1-8054-db65f30da641-kube-api-access-w2m5g\") pod \"d5c6add8-f1b7-4cd1-8054-db65f30da641\" (UID: \"d5c6add8-f1b7-4cd1-8054-db65f30da641\") " Jul 15 19:24:44.525973 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.525926 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5c6add8-f1b7-4cd1-8054-db65f30da641-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d5c6add8-f1b7-4cd1-8054-db65f30da641" (UID: "d5c6add8-f1b7-4cd1-8054-db65f30da641"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:24:44.539186 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.539132 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eeac26d-05e2-4c5f-bc7f-096b01077081-kube-api-access-rzlqs" (OuterVolumeSpecName: "kube-api-access-rzlqs") pod "7eeac26d-05e2-4c5f-bc7f-096b01077081" (UID: "7eeac26d-05e2-4c5f-bc7f-096b01077081"). InnerVolumeSpecName "kube-api-access-rzlqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:24:44.547327 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.547247 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5c6add8-f1b7-4cd1-8054-db65f30da641-kube-api-access-w2m5g" (OuterVolumeSpecName: "kube-api-access-w2m5g") pod "d5c6add8-f1b7-4cd1-8054-db65f30da641" (UID: "d5c6add8-f1b7-4cd1-8054-db65f30da641"). InnerVolumeSpecName "kube-api-access-w2m5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:24:44.617167 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.617137 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc"] Jul 15 19:24:44.619084 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:24:44.619058 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c14da53_96cd_494a_a713_cfac1aab77b5.slice/crio-d219b7043155cce247b4fcf081aaf9c4cf7ef6afa482dd69c9dfc7deec450e1d WatchSource:0}: Error finding container d219b7043155cce247b4fcf081aaf9c4cf7ef6afa482dd69c9dfc7deec450e1d: Status 404 returned error can't find the container with id d219b7043155cce247b4fcf081aaf9c4cf7ef6afa482dd69c9dfc7deec450e1d Jul 15 19:24:44.624321 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.624303 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzlqs\" (UniqueName: \"kubernetes.io/projected/7eeac26d-05e2-4c5f-bc7f-096b01077081-kube-api-access-rzlqs\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:24:44.624412 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.624324 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d5c6add8-f1b7-4cd1-8054-db65f30da641-extensions-socket-volume\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:24:44.624412 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.624334 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2m5g\" (UniqueName: \"kubernetes.io/projected/d5c6add8-f1b7-4cd1-8054-db65f30da641-kube-api-access-w2m5g\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:24:44.642760 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.642730 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz"] Jul 15 19:24:44.646494 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:24:44.646457 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d38c22_7196_4aa5_b2b4_91347feae83c.slice/crio-a516a81234ef776ff6b7dc02bef1ae48c0795fdae452b33518684ef2ba699ddd WatchSource:0}: Error finding container a516a81234ef776ff6b7dc02bef1ae48c0795fdae452b33518684ef2ba699ddd: Status 404 returned error can't find the container with id a516a81234ef776ff6b7dc02bef1ae48c0795fdae452b33518684ef2ba699ddd Jul 15 19:24:44.741417 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:44.741380 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:44.743274 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.743249 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" path="/var/lib/kubelet/pods/7eeac26d-05e2-4c5f-bc7f-096b01077081/volumes" Jul 15 19:24:44.743348 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:44.743306 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f858j\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:44.743663 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.743648 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" path="/var/lib/kubelet/pods/d5c6add8-f1b7-4cd1-8054-db65f30da641/volumes" Jul 15 19:24:44.892173 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.892144 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:44.892173 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.892170 2562 scope.go:117] "RemoveContainer" containerID="d84da15af49d559196eea25629ee104d55094ea1ea5125c8e76d059d41840ef1" Jul 15 19:24:44.894270 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.894236 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" event={"ID":"0c14da53-96cd-494a-a713-cfac1aab77b5","Type":"ContainerStarted","Data":"d785fc538e1a16b06b3dde57b4b7bd7ede13964c0d39b7c5c6a8987451b1654b"} Jul 15 19:24:44.894397 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.894299 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" event={"ID":"0c14da53-96cd-494a-a713-cfac1aab77b5","Type":"ContainerStarted","Data":"d219b7043155cce247b4fcf081aaf9c4cf7ef6afa482dd69c9dfc7deec450e1d"} Jul 15 19:24:44.894824 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:44.894789 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:44.894945 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.894848 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" Jul 15 19:24:44.896906 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.896611 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:44.899089 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.899059 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" event={"ID":"d43e74ae-4bec-44ec-a141-889949d3016b","Type":"ContainerStarted","Data":"9bfa7a70ec0373c9423718a626aad22385fc5eaef46d8cc86fc8a3f43b054e00"} Jul 15 19:24:44.901153 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.901126 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" event={"ID":"28d38c22-7196-4aa5-b2b4-91347feae83c","Type":"ContainerStarted","Data":"ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec"} Jul 15 19:24:44.901253 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.901173 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" event={"ID":"28d38c22-7196-4aa5-b2b4-91347feae83c","Type":"ContainerStarted","Data":"a516a81234ef776ff6b7dc02bef1ae48c0795fdae452b33518684ef2ba699ddd"} Jul 15 19:24:44.901301 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.901290 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:44.904517 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.904498 2562 scope.go:117] "RemoveContainer" containerID="42399c182e6aef96571f27b1d9f37c6ebce3afa4ecf28fa0d38c63152d171692" Jul 15 19:24:44.911966 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.911922 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" podStartSLOduration=14.911908735 podStartE2EDuration="14.911908735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:24:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:24:44.91067122 +0000 UTC m=+530.747375998" watchObservedRunningTime="2026-07-15 19:24:44.911908735 +0000 UTC m=+530.748613575" Jul 15 19:24:44.913113 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:44.913008 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:44.915141 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:44.915119 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-f48zq\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="d5c6add8-f1b7-4cd1-8054-db65f30da641" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f48zq" Jul 15 19:24:44.917095 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:44.917075 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-f858j\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="7eeac26d-05e2-4c5f-bc7f-096b01077081" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-f858j" Jul 15 19:24:44.935922 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.935876 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" podStartSLOduration=14.935862252 podStartE2EDuration="14.935862252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:24:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:24:44.934490467 +0000 UTC m=+530.771195247" watchObservedRunningTime="2026-07-15 19:24:44.935862252 +0000 UTC m=+530.772567028" Jul 15 19:24:44.948931 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:44.948886 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zx6jx" podStartSLOduration=1.012462852 podStartE2EDuration="24.948872944s" totalImagesPullingTime="23.936410092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:24:44.948125875 +0000 UTC m=+530.784830651" watchObservedRunningTime="2026-07-15 19:24:44.948872944 +0000 UTC m=+530.785577720" Jul 15 19:24:47.036715 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.036680 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz"] Jul 15 19:24:47.037159 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.036913 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" podUID="28d38c22-7196-4aa5-b2b4-91347feae83c" containerName="manager" containerID="cri-o://ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec" gracePeriod=10 Jul 15 19:24:47.334043 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.334011 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:47.448952 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.448921 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28d38c22-7196-4aa5-b2b4-91347feae83c-extensions-socket-volume\") pod \"28d38c22-7196-4aa5-b2b4-91347feae83c\" (UID: \"28d38c22-7196-4aa5-b2b4-91347feae83c\") " Jul 15 19:24:47.449134 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.449006 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zx5z\" (UniqueName: \"kubernetes.io/projected/28d38c22-7196-4aa5-b2b4-91347feae83c-kube-api-access-4zx5z\") pod \"28d38c22-7196-4aa5-b2b4-91347feae83c\" (UID: \"28d38c22-7196-4aa5-b2b4-91347feae83c\") " Jul 15 19:24:47.449374 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.449347 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d38c22-7196-4aa5-b2b4-91347feae83c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "28d38c22-7196-4aa5-b2b4-91347feae83c" (UID: "28d38c22-7196-4aa5-b2b4-91347feae83c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:24:47.451174 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.451148 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d38c22-7196-4aa5-b2b4-91347feae83c-kube-api-access-4zx5z" (OuterVolumeSpecName: "kube-api-access-4zx5z") pod "28d38c22-7196-4aa5-b2b4-91347feae83c" (UID: "28d38c22-7196-4aa5-b2b4-91347feae83c"). InnerVolumeSpecName "kube-api-access-4zx5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:24:47.549625 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.549553 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28d38c22-7196-4aa5-b2b4-91347feae83c-extensions-socket-volume\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:24:47.549625 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.549601 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zx5z\" (UniqueName: \"kubernetes.io/projected/28d38c22-7196-4aa5-b2b4-91347feae83c-kube-api-access-4zx5z\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:24:47.920824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.920788 2562 generic.go:356] "Generic (PLEG): container finished" podID="28d38c22-7196-4aa5-b2b4-91347feae83c" containerID="ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec" exitCode=0 Jul 15 19:24:47.921018 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.920836 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" Jul 15 19:24:47.921018 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.920874 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" event={"ID":"28d38c22-7196-4aa5-b2b4-91347feae83c","Type":"ContainerDied","Data":"ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec"} Jul 15 19:24:47.921018 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.920929 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz" event={"ID":"28d38c22-7196-4aa5-b2b4-91347feae83c","Type":"ContainerDied","Data":"a516a81234ef776ff6b7dc02bef1ae48c0795fdae452b33518684ef2ba699ddd"} Jul 15 19:24:47.921018 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.920951 2562 scope.go:117] "RemoveContainer" containerID="ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec" Jul 15 19:24:47.930353 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.930335 2562 scope.go:117] "RemoveContainer" containerID="ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec" Jul 15 19:24:47.930630 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:24:47.930613 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec\": container with ID starting with ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec not found: ID does not exist" containerID="ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec" Jul 15 19:24:47.930682 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.930639 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec"} err="failed to get container status \"ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec\": rpc error: code = NotFound desc = could not find container \"ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec\": container with ID starting with ea04d458c5afc58212ab369935ba4aaa039e6a9a227bd4f104965522f0ade0ec not found: ID does not exist" Jul 15 19:24:47.945283 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.945259 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz"] Jul 15 19:24:47.949268 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:47.949248 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-dgwdz"] Jul 15 19:24:48.742983 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:48.742950 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28d38c22-7196-4aa5-b2b4-91347feae83c" path="/var/lib/kubelet/pods/28d38c22-7196-4aa5-b2b4-91347feae83c/volumes" Jul 15 19:24:56.916421 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:24:56.916384 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wbrgc" Jul 15 19:25:03.453625 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.453590 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph"] Jul 15 19:25:03.454003 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.453903 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28d38c22-7196-4aa5-b2b4-91347feae83c" containerName="manager" Jul 15 19:25:03.454003 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.453913 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="28d38c22-7196-4aa5-b2b4-91347feae83c" containerName="manager" Jul 15 19:25:03.454003 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.453959 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="28d38c22-7196-4aa5-b2b4-91347feae83c" containerName="manager" Jul 15 19:25:03.472481 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.472457 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.475661 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.475641 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-lqtbj\"" Jul 15 19:25:03.491740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.491713 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph"] Jul 15 19:25:03.594937 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.594903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9de8a364-dd9e-47bb-9520-3a51688ff72f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.594937 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.594944 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.595148 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.594963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.595148 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.595010 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.595148 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.595078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.595148 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.595102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.595148 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.595123 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.595296 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.595190 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.595296 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.595226 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4z77\" (UniqueName: \"kubernetes.io/projected/9de8a364-dd9e-47bb-9520-3a51688ff72f-kube-api-access-v4z77\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.695879 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.695846 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696053 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.695889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696053 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.695915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696053 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.695931 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4z77\" (UniqueName: \"kubernetes.io/projected/9de8a364-dd9e-47bb-9520-3a51688ff72f-kube-api-access-v4z77\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696053 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.695991 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9de8a364-dd9e-47bb-9520-3a51688ff72f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696053 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.696021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696053 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.696048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696350 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.696074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696350 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.696108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696441 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.696413 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696514 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.696484 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696650 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.696523 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696650 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.696545 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.696770 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.696751 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9de8a364-dd9e-47bb-9520-3a51688ff72f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.698266 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.698252 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.698513 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.698497 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.703729 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.703678 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9de8a364-dd9e-47bb-9520-3a51688ff72f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.703993 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.703971 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4z77\" (UniqueName: \"kubernetes.io/projected/9de8a364-dd9e-47bb-9520-3a51688ff72f-kube-api-access-v4z77\") pod \"maas-default-gateway-openshift-default-56654dcf7-ts9ph\" (UID: \"9de8a364-dd9e-47bb-9520-3a51688ff72f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.782903 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.782869 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:03.917318 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.917292 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph"] Jul 15 19:25:03.919441 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:25:03.919394 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de8a364_dd9e_47bb_9520_3a51688ff72f.slice/crio-d28ad8bf869f85cdd168d8290af5fc3bfe9e51df1d9c845df14168ac1875b47c WatchSource:0}: Error finding container d28ad8bf869f85cdd168d8290af5fc3bfe9e51df1d9c845df14168ac1875b47c: Status 404 returned error can't find the container with id d28ad8bf869f85cdd168d8290af5fc3bfe9e51df1d9c845df14168ac1875b47c Jul 15 19:25:03.921673 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.921639 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:25:03.921733 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.921704 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:25:03.921768 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.921733 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:25:03.998116 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.998059 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" event={"ID":"9de8a364-dd9e-47bb-9520-3a51688ff72f","Type":"ContainerStarted","Data":"b5dc0cb270a6477c4ce4a016e6c9074cbebe41d576a94e0701fb5a9c4ca12607"} Jul 15 19:25:03.998248 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:03.998142 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" event={"ID":"9de8a364-dd9e-47bb-9520-3a51688ff72f","Type":"ContainerStarted","Data":"d28ad8bf869f85cdd168d8290af5fc3bfe9e51df1d9c845df14168ac1875b47c"} Jul 15 19:25:04.018662 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:04.018610 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" podStartSLOduration=1.018561442 podStartE2EDuration="1.018561442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:04.016103807 +0000 UTC m=+549.852808589" watchObservedRunningTime="2026-07-15 19:25:04.018561442 +0000 UTC m=+549.855266219" Jul 15 19:25:04.783041 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:04.783009 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:04.787817 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:04.787793 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:05.003197 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.003172 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:05.004038 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.004022 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:25:05.098031 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.097957 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2cpg"] Jul 15 19:25:05.101241 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.101226 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:05.103757 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.103734 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 19:25:05.111091 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.111069 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2cpg"] Jul 15 19:25:05.195675 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.195639 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2cpg"] Jul 15 19:25:05.210072 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.210047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kdkv\" (UniqueName: \"kubernetes.io/projected/fba8ce79-f2b7-4525-b960-237a766b4212-kube-api-access-4kdkv\") pod \"limitador-limitador-64c94fd64f-z2cpg\" (UID: \"fba8ce79-f2b7-4525-b960-237a766b4212\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:05.210213 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.210116 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fba8ce79-f2b7-4525-b960-237a766b4212-config-file\") pod \"limitador-limitador-64c94fd64f-z2cpg\" (UID: \"fba8ce79-f2b7-4525-b960-237a766b4212\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:05.311324 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.311296 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fba8ce79-f2b7-4525-b960-237a766b4212-config-file\") pod \"limitador-limitador-64c94fd64f-z2cpg\" (UID: \"fba8ce79-f2b7-4525-b960-237a766b4212\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:05.311474 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.311354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kdkv\" (UniqueName: \"kubernetes.io/projected/fba8ce79-f2b7-4525-b960-237a766b4212-kube-api-access-4kdkv\") pod \"limitador-limitador-64c94fd64f-z2cpg\" (UID: \"fba8ce79-f2b7-4525-b960-237a766b4212\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:05.311935 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.311917 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fba8ce79-f2b7-4525-b960-237a766b4212-config-file\") pod \"limitador-limitador-64c94fd64f-z2cpg\" (UID: \"fba8ce79-f2b7-4525-b960-237a766b4212\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:05.319739 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.319710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kdkv\" (UniqueName: \"kubernetes.io/projected/fba8ce79-f2b7-4525-b960-237a766b4212-kube-api-access-4kdkv\") pod \"limitador-limitador-64c94fd64f-z2cpg\" (UID: \"fba8ce79-f2b7-4525-b960-237a766b4212\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:05.411734 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.411701 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:05.550390 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:05.550366 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2cpg"] Jul 15 19:25:05.552625 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:25:05.552591 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba8ce79_f2b7_4525_b960_237a766b4212.slice/crio-c8a7378d7f9a1efd931c0f5c67cc22f4ea4fbd1f1a39da8e75a5eb9dc6945f07 WatchSource:0}: Error finding container c8a7378d7f9a1efd931c0f5c67cc22f4ea4fbd1f1a39da8e75a5eb9dc6945f07: Status 404 returned error can't find the container with id c8a7378d7f9a1efd931c0f5c67cc22f4ea4fbd1f1a39da8e75a5eb9dc6945f07 Jul 15 19:25:06.010389 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.010349 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" event={"ID":"fba8ce79-f2b7-4525-b960-237a766b4212","Type":"ContainerStarted","Data":"c8a7378d7f9a1efd931c0f5c67cc22f4ea4fbd1f1a39da8e75a5eb9dc6945f07"} Jul 15 19:25:06.075997 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.075961 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-djcxf"] Jul 15 19:25:06.093644 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.093614 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-djcxf"] Jul 15 19:25:06.093804 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.093661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-djcxf" Jul 15 19:25:06.096279 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.096259 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r9c97\"" Jul 15 19:25:06.219397 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.219358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxbt6\" (UniqueName: \"kubernetes.io/projected/73f6b294-b0d9-410d-a710-06e03b250799-kube-api-access-pxbt6\") pod \"authorino-75d556bd77-djcxf\" (UID: \"73f6b294-b0d9-410d-a710-06e03b250799\") " pod="kuadrant-system/authorino-75d556bd77-djcxf" Jul 15 19:25:06.320889 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.320818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxbt6\" (UniqueName: \"kubernetes.io/projected/73f6b294-b0d9-410d-a710-06e03b250799-kube-api-access-pxbt6\") pod \"authorino-75d556bd77-djcxf\" (UID: \"73f6b294-b0d9-410d-a710-06e03b250799\") " pod="kuadrant-system/authorino-75d556bd77-djcxf" Jul 15 19:25:06.330047 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.330022 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxbt6\" (UniqueName: \"kubernetes.io/projected/73f6b294-b0d9-410d-a710-06e03b250799-kube-api-access-pxbt6\") pod \"authorino-75d556bd77-djcxf\" (UID: \"73f6b294-b0d9-410d-a710-06e03b250799\") " pod="kuadrant-system/authorino-75d556bd77-djcxf" Jul 15 19:25:06.403914 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.403883 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-djcxf" Jul 15 19:25:06.569403 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:06.569365 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-djcxf"] Jul 15 19:25:07.017851 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:07.017812 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-djcxf" event={"ID":"73f6b294-b0d9-410d-a710-06e03b250799","Type":"ContainerStarted","Data":"23508c83e8dcc5da7b0983f3158f90bd4c67bfc2363930054d599a948c38a9d0"} Jul 15 19:25:10.041201 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:10.041163 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" event={"ID":"fba8ce79-f2b7-4525-b960-237a766b4212","Type":"ContainerStarted","Data":"b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf"} Jul 15 19:25:10.041695 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:10.041333 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:10.061819 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:10.061477 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" podStartSLOduration=1.040432329 podStartE2EDuration="5.061455778s" totalImagesPullingTime="4.021023449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:10.059974925 +0000 UTC m=+555.896679704" watchObservedRunningTime="2026-07-15 19:25:10.061455778 +0000 UTC m=+555.898160555" Jul 15 19:25:13.059708 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:13.059667 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-djcxf" event={"ID":"73f6b294-b0d9-410d-a710-06e03b250799","Type":"ContainerStarted","Data":"39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1"} Jul 15 19:25:13.075346 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:13.075300 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-djcxf" podStartSLOduration=1.481224362 podStartE2EDuration="7.075283698s" totalImagesPullingTime="5.594059336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:13.074112774 +0000 UTC m=+558.910817552" watchObservedRunningTime="2026-07-15 19:25:13.075283698 +0000 UTC m=+558.911988521" Jul 15 19:25:20.286966 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.286931 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2cpg"] Jul 15 19:25:20.287460 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.287121 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" podUID="fba8ce79-f2b7-4525-b960-237a766b4212" containerName="limitador" containerID="cri-o://b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf" gracePeriod=30 Jul 15 19:25:20.287857 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.287834 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:20.825319 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.825294 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:20.853368 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.853297 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fba8ce79-f2b7-4525-b960-237a766b4212-config-file\") pod \"fba8ce79-f2b7-4525-b960-237a766b4212\" (UID: \"fba8ce79-f2b7-4525-b960-237a766b4212\") " Jul 15 19:25:20.853486 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.853371 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kdkv\" (UniqueName: \"kubernetes.io/projected/fba8ce79-f2b7-4525-b960-237a766b4212-kube-api-access-4kdkv\") pod \"fba8ce79-f2b7-4525-b960-237a766b4212\" (UID: \"fba8ce79-f2b7-4525-b960-237a766b4212\") " Jul 15 19:25:20.853556 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.853537 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fba8ce79-f2b7-4525-b960-237a766b4212-config-file" (OuterVolumeSpecName: "config-file") pod "fba8ce79-f2b7-4525-b960-237a766b4212" (UID: "fba8ce79-f2b7-4525-b960-237a766b4212"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:25:20.853671 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.853659 2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fba8ce79-f2b7-4525-b960-237a766b4212-config-file\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:25:20.855668 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.855636 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fba8ce79-f2b7-4525-b960-237a766b4212-kube-api-access-4kdkv" (OuterVolumeSpecName: "kube-api-access-4kdkv") pod "fba8ce79-f2b7-4525-b960-237a766b4212" (UID: "fba8ce79-f2b7-4525-b960-237a766b4212"). InnerVolumeSpecName "kube-api-access-4kdkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:25:20.954909 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:20.954878 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kdkv\" (UniqueName: \"kubernetes.io/projected/fba8ce79-f2b7-4525-b960-237a766b4212-kube-api-access-4kdkv\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:25:21.101657 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:21.101622 2562 generic.go:356] "Generic (PLEG): container finished" podID="fba8ce79-f2b7-4525-b960-237a766b4212" containerID="b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf" exitCode=0 Jul 15 19:25:21.101825 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:21.101655 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" event={"ID":"fba8ce79-f2b7-4525-b960-237a766b4212","Type":"ContainerDied","Data":"b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf"} Jul 15 19:25:21.101825 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:21.101674 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" Jul 15 19:25:21.101825 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:21.101709 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z2cpg" event={"ID":"fba8ce79-f2b7-4525-b960-237a766b4212","Type":"ContainerDied","Data":"c8a7378d7f9a1efd931c0f5c67cc22f4ea4fbd1f1a39da8e75a5eb9dc6945f07"} Jul 15 19:25:21.101825 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:21.101731 2562 scope.go:117] "RemoveContainer" containerID="b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf" Jul 15 19:25:21.110469 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:21.110450 2562 scope.go:117] "RemoveContainer" containerID="b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf" Jul 15 19:25:21.110755 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:25:21.110739 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf\": container with ID starting with b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf not found: ID does not exist" containerID="b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf" Jul 15 19:25:21.110828 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:21.110763 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf"} err="failed to get container status \"b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf\": rpc error: code = NotFound desc = could not find container \"b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf\": container with ID starting with b4c8fbfeaf191d5db58acad3a2799e1884302b36be109bde5ce96aba4d6a90cf not found: ID does not exist" Jul 15 19:25:21.122306 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:21.122280 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2cpg"] Jul 15 19:25:21.129449 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:21.129429 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z2cpg"] Jul 15 19:25:22.743626 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:22.743591 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fba8ce79-f2b7-4525-b960-237a766b4212" path="/var/lib/kubelet/pods/fba8ce79-f2b7-4525-b960-237a766b4212/volumes" Jul 15 19:25:26.770174 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.770142 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8"] Jul 15 19:25:26.770549 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.770438 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fba8ce79-f2b7-4525-b960-237a766b4212" containerName="limitador" Jul 15 19:25:26.770549 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.770448 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fba8ce79-f2b7-4525-b960-237a766b4212" containerName="limitador" Jul 15 19:25:26.770549 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.770510 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fba8ce79-f2b7-4525-b960-237a766b4212" containerName="limitador" Jul 15 19:25:26.774240 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.774223 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:26.777041 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.777012 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 15 19:25:26.777041 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.777024 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-vg7pv\"" Jul 15 19:25:26.777041 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.777014 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 19:25:26.779039 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.779023 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 19:25:26.783792 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.783772 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8"] Jul 15 19:25:26.801534 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.801505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxb46\" (UniqueName: \"kubernetes.io/projected/4ac389d6-1a82-4453-b0b7-8909bab06a13-kube-api-access-gxb46\") pod \"postgres-69fb4594cf-wnqc8\" (UID: \"4ac389d6-1a82-4453-b0b7-8909bab06a13\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:26.801682 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.801635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4ac389d6-1a82-4453-b0b7-8909bab06a13-data\") pod \"postgres-69fb4594cf-wnqc8\" (UID: \"4ac389d6-1a82-4453-b0b7-8909bab06a13\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:26.903067 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.903036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4ac389d6-1a82-4453-b0b7-8909bab06a13-data\") pod \"postgres-69fb4594cf-wnqc8\" (UID: \"4ac389d6-1a82-4453-b0b7-8909bab06a13\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:26.903234 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.903106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxb46\" (UniqueName: \"kubernetes.io/projected/4ac389d6-1a82-4453-b0b7-8909bab06a13-kube-api-access-gxb46\") pod \"postgres-69fb4594cf-wnqc8\" (UID: \"4ac389d6-1a82-4453-b0b7-8909bab06a13\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:26.903481 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.903459 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4ac389d6-1a82-4453-b0b7-8909bab06a13-data\") pod \"postgres-69fb4594cf-wnqc8\" (UID: \"4ac389d6-1a82-4453-b0b7-8909bab06a13\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:26.912529 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:26.912491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxb46\" (UniqueName: \"kubernetes.io/projected/4ac389d6-1a82-4453-b0b7-8909bab06a13-kube-api-access-gxb46\") pod \"postgres-69fb4594cf-wnqc8\" (UID: \"4ac389d6-1a82-4453-b0b7-8909bab06a13\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:27.084427 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:27.084351 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:27.215221 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:27.215196 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8"] Jul 15 19:25:27.217484 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:25:27.217458 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ac389d6_1a82_4453_b0b7_8909bab06a13.slice/crio-624cc835bc57e7e3ef1a92c3703c846882cca0fdf853da16254365f87edc7838 WatchSource:0}: Error finding container 624cc835bc57e7e3ef1a92c3703c846882cca0fdf853da16254365f87edc7838: Status 404 returned error can't find the container with id 624cc835bc57e7e3ef1a92c3703c846882cca0fdf853da16254365f87edc7838 Jul 15 19:25:28.139643 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:28.139596 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" event={"ID":"4ac389d6-1a82-4453-b0b7-8909bab06a13","Type":"ContainerStarted","Data":"624cc835bc57e7e3ef1a92c3703c846882cca0fdf853da16254365f87edc7838"} Jul 15 19:25:33.170690 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:33.170656 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" event={"ID":"4ac389d6-1a82-4453-b0b7-8909bab06a13","Type":"ContainerStarted","Data":"e0c4ee18926e697b0b5402e615e382131eff994d30c945bfe4815212aced26c0"} Jul 15 19:25:33.171080 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:33.170772 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:33.187679 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:33.187635 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" podStartSLOduration=2.113077936 podStartE2EDuration="7.187622288s" totalImagesPullingTime="5.074544352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:33.185343971 +0000 UTC m=+579.022048744" watchObservedRunningTime="2026-07-15 19:25:33.187622288 +0000 UTC m=+579.024327064" Jul 15 19:25:39.205095 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:39.205063 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wnqc8" Jul 15 19:25:39.775066 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:39.775032 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-x6xk2"] Jul 15 19:25:39.780978 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:39.780959 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" Jul 15 19:25:39.785174 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:39.785146 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-x6xk2"] Jul 15 19:25:39.918250 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:39.918213 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbktj\" (UniqueName: \"kubernetes.io/projected/2529c730-86ce-4005-bf17-5cfb076990fb-kube-api-access-sbktj\") pod \"authorino-5d4b67c446-x6xk2\" (UID: \"2529c730-86ce-4005-bf17-5cfb076990fb\") " pod="kuadrant-system/authorino-5d4b67c446-x6xk2" Jul 15 19:25:40.019137 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.019093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbktj\" (UniqueName: \"kubernetes.io/projected/2529c730-86ce-4005-bf17-5cfb076990fb-kube-api-access-sbktj\") pod \"authorino-5d4b67c446-x6xk2\" (UID: \"2529c730-86ce-4005-bf17-5cfb076990fb\") " pod="kuadrant-system/authorino-5d4b67c446-x6xk2" Jul 15 19:25:40.029869 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.029801 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbktj\" (UniqueName: \"kubernetes.io/projected/2529c730-86ce-4005-bf17-5cfb076990fb-kube-api-access-sbktj\") pod \"authorino-5d4b67c446-x6xk2\" (UID: \"2529c730-86ce-4005-bf17-5cfb076990fb\") " pod="kuadrant-system/authorino-5d4b67c446-x6xk2" Jul 15 19:25:40.041305 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.041269 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-x6xk2"] Jul 15 19:25:40.041601 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.041570 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" Jul 15 19:25:40.062134 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.062110 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-848cc446c-q8n8h"] Jul 15 19:25:40.066851 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.066830 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-848cc446c-q8n8h" Jul 15 19:25:40.070436 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.070408 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-848cc446c-q8n8h"] Jul 15 19:25:40.120371 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.120329 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkzlf\" (UniqueName: \"kubernetes.io/projected/1b08e31a-2919-4cbd-ae94-c5aedca53c50-kube-api-access-zkzlf\") pod \"authorino-848cc446c-q8n8h\" (UID: \"1b08e31a-2919-4cbd-ae94-c5aedca53c50\") " pod="kuadrant-system/authorino-848cc446c-q8n8h" Jul 15 19:25:40.173837 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.173813 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-x6xk2"] Jul 15 19:25:40.175528 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:25:40.175501 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2529c730_86ce_4005_bf17_5cfb076990fb.slice/crio-6c4584ebce98bb11f4bd05b5154626f2c77d5f993cd2e13481c7ee13e8e5c7b5 WatchSource:0}: Error finding container 6c4584ebce98bb11f4bd05b5154626f2c77d5f993cd2e13481c7ee13e8e5c7b5: Status 404 returned error can't find the container with id 6c4584ebce98bb11f4bd05b5154626f2c77d5f993cd2e13481c7ee13e8e5c7b5 Jul 15 19:25:40.212199 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.212169 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" event={"ID":"2529c730-86ce-4005-bf17-5cfb076990fb","Type":"ContainerStarted","Data":"6c4584ebce98bb11f4bd05b5154626f2c77d5f993cd2e13481c7ee13e8e5c7b5"} Jul 15 19:25:40.221534 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.221511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkzlf\" (UniqueName: \"kubernetes.io/projected/1b08e31a-2919-4cbd-ae94-c5aedca53c50-kube-api-access-zkzlf\") pod \"authorino-848cc446c-q8n8h\" (UID: \"1b08e31a-2919-4cbd-ae94-c5aedca53c50\") " pod="kuadrant-system/authorino-848cc446c-q8n8h" Jul 15 19:25:40.230426 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.230406 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkzlf\" (UniqueName: \"kubernetes.io/projected/1b08e31a-2919-4cbd-ae94-c5aedca53c50-kube-api-access-zkzlf\") pod \"authorino-848cc446c-q8n8h\" (UID: \"1b08e31a-2919-4cbd-ae94-c5aedca53c50\") " pod="kuadrant-system/authorino-848cc446c-q8n8h" Jul 15 19:25:40.377657 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.377548 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-848cc446c-q8n8h" Jul 15 19:25:40.508652 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:40.508629 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-848cc446c-q8n8h"] Jul 15 19:25:40.510262 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:25:40.510231 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b08e31a_2919_4cbd_ae94_c5aedca53c50.slice/crio-54ab3970484304d9dac39412e6a4bf1b03dcc1495f08e757135b629e8f43e8ef WatchSource:0}: Error finding container 54ab3970484304d9dac39412e6a4bf1b03dcc1495f08e757135b629e8f43e8ef: Status 404 returned error can't find the container with id 54ab3970484304d9dac39412e6a4bf1b03dcc1495f08e757135b629e8f43e8ef Jul 15 19:25:41.218739 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.218697 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" event={"ID":"2529c730-86ce-4005-bf17-5cfb076990fb","Type":"ContainerStarted","Data":"ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c"} Jul 15 19:25:41.219201 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.218751 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" podUID="2529c730-86ce-4005-bf17-5cfb076990fb" containerName="authorino" containerID="cri-o://ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c" gracePeriod=30 Jul 15 19:25:41.220470 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.220442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-848cc446c-q8n8h" event={"ID":"1b08e31a-2919-4cbd-ae94-c5aedca53c50","Type":"ContainerStarted","Data":"a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a"} Jul 15 19:25:41.220628 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.220483 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-848cc446c-q8n8h" event={"ID":"1b08e31a-2919-4cbd-ae94-c5aedca53c50","Type":"ContainerStarted","Data":"54ab3970484304d9dac39412e6a4bf1b03dcc1495f08e757135b629e8f43e8ef"} Jul 15 19:25:41.234988 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.234944 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" podStartSLOduration=1.817538044 podStartE2EDuration="2.234929961s" totalImagesPullingTime="417.391917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:41.233632141 +0000 UTC m=+587.070336910" watchObservedRunningTime="2026-07-15 19:25:41.234929961 +0000 UTC m=+587.071634737" Jul 15 19:25:41.249034 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.248985 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-848cc446c-q8n8h" podStartSLOduration=0.937787795 podStartE2EDuration="1.248967082s" totalImagesPullingTime="311.179287ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:41.246873361 +0000 UTC m=+587.083578137" watchObservedRunningTime="2026-07-15 19:25:41.248967082 +0000 UTC m=+587.085671859" Jul 15 19:25:41.275427 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.275374 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-djcxf"] Jul 15 19:25:41.275624 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.275607 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-djcxf" podUID="73f6b294-b0d9-410d-a710-06e03b250799" containerName="authorino" containerID="cri-o://39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1" gracePeriod=30 Jul 15 19:25:41.527801 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.527779 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" Jul 15 19:25:41.530706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.530683 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-djcxf" Jul 15 19:25:41.633671 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.633640 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbktj\" (UniqueName: \"kubernetes.io/projected/2529c730-86ce-4005-bf17-5cfb076990fb-kube-api-access-sbktj\") pod \"2529c730-86ce-4005-bf17-5cfb076990fb\" (UID: \"2529c730-86ce-4005-bf17-5cfb076990fb\") " Jul 15 19:25:41.633871 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.633685 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxbt6\" (UniqueName: \"kubernetes.io/projected/73f6b294-b0d9-410d-a710-06e03b250799-kube-api-access-pxbt6\") pod \"73f6b294-b0d9-410d-a710-06e03b250799\" (UID: \"73f6b294-b0d9-410d-a710-06e03b250799\") " Jul 15 19:25:41.635792 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.635758 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2529c730-86ce-4005-bf17-5cfb076990fb-kube-api-access-sbktj" (OuterVolumeSpecName: "kube-api-access-sbktj") pod "2529c730-86ce-4005-bf17-5cfb076990fb" (UID: "2529c730-86ce-4005-bf17-5cfb076990fb"). InnerVolumeSpecName "kube-api-access-sbktj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:25:41.635886 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.635849 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73f6b294-b0d9-410d-a710-06e03b250799-kube-api-access-pxbt6" (OuterVolumeSpecName: "kube-api-access-pxbt6") pod "73f6b294-b0d9-410d-a710-06e03b250799" (UID: "73f6b294-b0d9-410d-a710-06e03b250799"). InnerVolumeSpecName "kube-api-access-pxbt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:25:41.734769 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.734692 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbktj\" (UniqueName: \"kubernetes.io/projected/2529c730-86ce-4005-bf17-5cfb076990fb-kube-api-access-sbktj\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:25:41.734769 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:41.734723 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxbt6\" (UniqueName: \"kubernetes.io/projected/73f6b294-b0d9-410d-a710-06e03b250799-kube-api-access-pxbt6\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:25:42.226666 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.226632 2562 generic.go:356] "Generic (PLEG): container finished" podID="73f6b294-b0d9-410d-a710-06e03b250799" containerID="39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1" exitCode=0 Jul 15 19:25:42.227073 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.226674 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-djcxf" Jul 15 19:25:42.227073 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.226701 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-djcxf" event={"ID":"73f6b294-b0d9-410d-a710-06e03b250799","Type":"ContainerDied","Data":"39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1"} Jul 15 19:25:42.227073 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.226739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-djcxf" event={"ID":"73f6b294-b0d9-410d-a710-06e03b250799","Type":"ContainerDied","Data":"23508c83e8dcc5da7b0983f3158f90bd4c67bfc2363930054d599a948c38a9d0"} Jul 15 19:25:42.227073 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.226759 2562 scope.go:117] "RemoveContainer" containerID="39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1" Jul 15 19:25:42.228123 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.228100 2562 generic.go:356] "Generic (PLEG): container finished" podID="2529c730-86ce-4005-bf17-5cfb076990fb" containerID="ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c" exitCode=0 Jul 15 19:25:42.228211 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.228137 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" Jul 15 19:25:42.228211 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.228181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" event={"ID":"2529c730-86ce-4005-bf17-5cfb076990fb","Type":"ContainerDied","Data":"ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c"} Jul 15 19:25:42.228293 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.228221 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-x6xk2" event={"ID":"2529c730-86ce-4005-bf17-5cfb076990fb","Type":"ContainerDied","Data":"6c4584ebce98bb11f4bd05b5154626f2c77d5f993cd2e13481c7ee13e8e5c7b5"} Jul 15 19:25:42.236924 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.236904 2562 scope.go:117] "RemoveContainer" containerID="39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1" Jul 15 19:25:42.237205 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:25:42.237186 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1\": container with ID starting with 39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1 not found: ID does not exist" containerID="39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1" Jul 15 19:25:42.237254 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.237214 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1"} err="failed to get container status \"39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1\": rpc error: code = NotFound desc = could not find container \"39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1\": container with ID starting with 39a198d60a4da24e601c3861e275541012727841d05b903a0302978de41397a1 not found: ID does not exist" Jul 15 19:25:42.237254 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.237231 2562 scope.go:117] "RemoveContainer" containerID="ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c" Jul 15 19:25:42.245622 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.245441 2562 scope.go:117] "RemoveContainer" containerID="ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c" Jul 15 19:25:42.245842 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:25:42.245817 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c\": container with ID starting with ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c not found: ID does not exist" containerID="ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c" Jul 15 19:25:42.245891 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.245853 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c"} err="failed to get container status \"ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c\": rpc error: code = NotFound desc = could not find container \"ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c\": container with ID starting with ada05abc48a4464350fac98de63d6d6605b80312ad149612ebf7a1907e2df53c not found: ID does not exist" Jul 15 19:25:42.248075 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.248055 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-djcxf"] Jul 15 19:25:42.249956 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.249939 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-djcxf"] Jul 15 19:25:42.259880 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.259861 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-x6xk2"] Jul 15 19:25:42.265032 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.265011 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-x6xk2"] Jul 15 19:25:42.743771 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.743739 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2529c730-86ce-4005-bf17-5cfb076990fb" path="/var/lib/kubelet/pods/2529c730-86ce-4005-bf17-5cfb076990fb/volumes" Jul 15 19:25:42.744089 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:42.744075 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73f6b294-b0d9-410d-a710-06e03b250799" path="/var/lib/kubelet/pods/73f6b294-b0d9-410d-a710-06e03b250799/volumes" Jul 15 19:25:43.342877 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:43.342841 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-848cc446c-q8n8h"] Jul 15 19:25:43.343329 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:43.342990 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-848cc446c-q8n8h" podUID="1b08e31a-2919-4cbd-ae94-c5aedca53c50" containerName="authorino" containerID="cri-o://a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a" gracePeriod=30 Jul 15 19:25:43.626286 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:43.626225 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-848cc446c-q8n8h" Jul 15 19:25:43.752915 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:43.752887 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkzlf\" (UniqueName: \"kubernetes.io/projected/1b08e31a-2919-4cbd-ae94-c5aedca53c50-kube-api-access-zkzlf\") pod \"1b08e31a-2919-4cbd-ae94-c5aedca53c50\" (UID: \"1b08e31a-2919-4cbd-ae94-c5aedca53c50\") " Jul 15 19:25:43.755046 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:43.755006 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b08e31a-2919-4cbd-ae94-c5aedca53c50-kube-api-access-zkzlf" (OuterVolumeSpecName: "kube-api-access-zkzlf") pod "1b08e31a-2919-4cbd-ae94-c5aedca53c50" (UID: "1b08e31a-2919-4cbd-ae94-c5aedca53c50"). InnerVolumeSpecName "kube-api-access-zkzlf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:25:43.854462 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:43.854420 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkzlf\" (UniqueName: \"kubernetes.io/projected/1b08e31a-2919-4cbd-ae94-c5aedca53c50-kube-api-access-zkzlf\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:25:44.240990 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.240952 2562 generic.go:356] "Generic (PLEG): container finished" podID="1b08e31a-2919-4cbd-ae94-c5aedca53c50" containerID="a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a" exitCode=0 Jul 15 19:25:44.240990 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.240994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-848cc446c-q8n8h" event={"ID":"1b08e31a-2919-4cbd-ae94-c5aedca53c50","Type":"ContainerDied","Data":"a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a"} Jul 15 19:25:44.241193 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.241027 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-848cc446c-q8n8h" event={"ID":"1b08e31a-2919-4cbd-ae94-c5aedca53c50","Type":"ContainerDied","Data":"54ab3970484304d9dac39412e6a4bf1b03dcc1495f08e757135b629e8f43e8ef"} Jul 15 19:25:44.241193 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.241043 2562 scope.go:117] "RemoveContainer" containerID="a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a" Jul 15 19:25:44.241193 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.240997 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-848cc446c-q8n8h" Jul 15 19:25:44.251644 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.251612 2562 scope.go:117] "RemoveContainer" containerID="a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a" Jul 15 19:25:44.252328 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:25:44.252288 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a\": container with ID starting with a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a not found: ID does not exist" containerID="a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a" Jul 15 19:25:44.252429 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.252335 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a"} err="failed to get container status \"a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a\": rpc error: code = NotFound desc = could not find container \"a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a\": container with ID starting with a1a59deb6802608a6f6d1fc02ea0e52c711405813f1071034ff8c4cffb44bd7a not found: ID does not exist" Jul 15 19:25:44.265379 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.265351 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-848cc446c-q8n8h"] Jul 15 19:25:44.267551 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.267531 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-848cc446c-q8n8h"] Jul 15 19:25:44.743989 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:44.743955 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b08e31a-2919-4cbd-ae94-c5aedca53c50" path="/var/lib/kubelet/pods/1b08e31a-2919-4cbd-ae94-c5aedca53c50/volumes" Jul 15 19:25:46.186315 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186282 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58794c5b99-cz78h"] Jul 15 19:25:46.186776 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186757 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b08e31a-2919-4cbd-ae94-c5aedca53c50" containerName="authorino" Jul 15 19:25:46.186824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186781 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b08e31a-2919-4cbd-ae94-c5aedca53c50" containerName="authorino" Jul 15 19:25:46.186824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186813 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73f6b294-b0d9-410d-a710-06e03b250799" containerName="authorino" Jul 15 19:25:46.186824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186822 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="73f6b294-b0d9-410d-a710-06e03b250799" containerName="authorino" Jul 15 19:25:46.186926 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186841 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2529c730-86ce-4005-bf17-5cfb076990fb" containerName="authorino" Jul 15 19:25:46.186926 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186849 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2529c730-86ce-4005-bf17-5cfb076990fb" containerName="authorino" Jul 15 19:25:46.186993 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186935 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b08e31a-2919-4cbd-ae94-c5aedca53c50" containerName="authorino" Jul 15 19:25:46.186993 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186950 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="73f6b294-b0d9-410d-a710-06e03b250799" containerName="authorino" Jul 15 19:25:46.186993 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.186960 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2529c730-86ce-4005-bf17-5cfb076990fb" containerName="authorino" Jul 15 19:25:46.283911 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.283877 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58794c5b99-cz78h"] Jul 15 19:25:46.284095 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.283973 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:25:46.288249 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.288228 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2d9mb\"" Jul 15 19:25:46.288381 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.288229 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 19:25:46.288381 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.288343 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 19:25:46.374014 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.373977 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cb8df54d8-j6lwc"] Jul 15 19:25:46.376995 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.376974 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nbwg\" (UniqueName: \"kubernetes.io/projected/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-kube-api-access-7nbwg\") pod \"maas-controller-58794c5b99-cz78h\" (UID: \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\") " pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:25:46.377076 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.377032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-webhook-cert\") pod \"maas-controller-58794c5b99-cz78h\" (UID: \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\") " pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:25:46.387882 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.387859 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb8df54d8-j6lwc"] Jul 15 19:25:46.388007 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.387948 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:25:46.478256 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.478179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nbwg\" (UniqueName: \"kubernetes.io/projected/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-kube-api-access-7nbwg\") pod \"maas-controller-58794c5b99-cz78h\" (UID: \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\") " pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:25:46.478256 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.478240 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8ed2272-962a-4cfa-b843-a40349dc4393-webhook-cert\") pod \"maas-controller-6cb8df54d8-j6lwc\" (UID: \"b8ed2272-962a-4cfa-b843-a40349dc4393\") " pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:25:46.478457 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.478264 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-webhook-cert\") pod \"maas-controller-58794c5b99-cz78h\" (UID: \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\") " pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:25:46.478457 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.478294 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs6nm\" (UniqueName: \"kubernetes.io/projected/b8ed2272-962a-4cfa-b843-a40349dc4393-kube-api-access-rs6nm\") pod \"maas-controller-6cb8df54d8-j6lwc\" (UID: \"b8ed2272-962a-4cfa-b843-a40349dc4393\") " pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:25:46.480929 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.480896 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-webhook-cert\") pod \"maas-controller-58794c5b99-cz78h\" (UID: \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\") " pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:25:46.487984 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.487955 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nbwg\" (UniqueName: \"kubernetes.io/projected/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-kube-api-access-7nbwg\") pod \"maas-controller-58794c5b99-cz78h\" (UID: \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\") " pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:25:46.579602 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.579532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8ed2272-962a-4cfa-b843-a40349dc4393-webhook-cert\") pod \"maas-controller-6cb8df54d8-j6lwc\" (UID: \"b8ed2272-962a-4cfa-b843-a40349dc4393\") " pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:25:46.579786 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.579632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs6nm\" (UniqueName: \"kubernetes.io/projected/b8ed2272-962a-4cfa-b843-a40349dc4393-kube-api-access-rs6nm\") pod \"maas-controller-6cb8df54d8-j6lwc\" (UID: \"b8ed2272-962a-4cfa-b843-a40349dc4393\") " pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:25:46.582045 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.582022 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8ed2272-962a-4cfa-b843-a40349dc4393-webhook-cert\") pod \"maas-controller-6cb8df54d8-j6lwc\" (UID: \"b8ed2272-962a-4cfa-b843-a40349dc4393\") " pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:25:46.589012 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.588987 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs6nm\" (UniqueName: \"kubernetes.io/projected/b8ed2272-962a-4cfa-b843-a40349dc4393-kube-api-access-rs6nm\") pod \"maas-controller-6cb8df54d8-j6lwc\" (UID: \"b8ed2272-962a-4cfa-b843-a40349dc4393\") " pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:25:46.593634 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.593614 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:25:46.700989 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.700918 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:25:46.743353 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.743329 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58794c5b99-cz78h"] Jul 15 19:25:46.744882 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:25:46.744852 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17cf68f2_440c_4bd2_9d88_39e1a04aed2d.slice/crio-fa85a67a0bc6cb033d188ff153fbb24cb29b819420b4885dd79e2f6243b2463c WatchSource:0}: Error finding container fa85a67a0bc6cb033d188ff153fbb24cb29b819420b4885dd79e2f6243b2463c: Status 404 returned error can't find the container with id fa85a67a0bc6cb033d188ff153fbb24cb29b819420b4885dd79e2f6243b2463c Jul 15 19:25:46.853012 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:46.852989 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb8df54d8-j6lwc"] Jul 15 19:25:46.855323 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:25:46.855297 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ed2272_962a_4cfa_b843_a40349dc4393.slice/crio-d4f6114a079e3207e17768ad071281c20ef48e3902467c0f2e92dc8b71dce3d7 WatchSource:0}: Error finding container d4f6114a079e3207e17768ad071281c20ef48e3902467c0f2e92dc8b71dce3d7: Status 404 returned error can't find the container with id d4f6114a079e3207e17768ad071281c20ef48e3902467c0f2e92dc8b71dce3d7 Jul 15 19:25:47.257273 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:47.257242 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" event={"ID":"b8ed2272-962a-4cfa-b843-a40349dc4393","Type":"ContainerStarted","Data":"d4f6114a079e3207e17768ad071281c20ef48e3902467c0f2e92dc8b71dce3d7"} Jul 15 19:25:47.258510 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:47.258488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58794c5b99-cz78h" event={"ID":"17cf68f2-440c-4bd2-9d88-39e1a04aed2d","Type":"ContainerStarted","Data":"fa85a67a0bc6cb033d188ff153fbb24cb29b819420b4885dd79e2f6243b2463c"} Jul 15 19:25:50.284398 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:50.284354 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" event={"ID":"b8ed2272-962a-4cfa-b843-a40349dc4393","Type":"ContainerStarted","Data":"bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a"} Jul 15 19:25:50.284858 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:50.284461 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:25:50.285938 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:50.285913 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58794c5b99-cz78h" event={"ID":"17cf68f2-440c-4bd2-9d88-39e1a04aed2d","Type":"ContainerStarted","Data":"a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034"} Jul 15 19:25:50.286072 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:50.286055 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:25:50.306128 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:50.306076 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" podStartSLOduration=1.72314548 podStartE2EDuration="4.306062911s" totalImagesPullingTime="2.582917431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:50.303877308 +0000 UTC m=+596.140582084" watchObservedRunningTime="2026-07-15 19:25:50.306062911 +0000 UTC m=+596.142767687" Jul 15 19:25:50.323058 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:50.323009 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58794c5b99-cz78h" podStartSLOduration=1.6394808410000001 podStartE2EDuration="4.322994213s" totalImagesPullingTime="2.683513372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:50.321436297 +0000 UTC m=+596.158141073" watchObservedRunningTime="2026-07-15 19:25:50.322994213 +0000 UTC m=+596.159698990" Jul 15 19:25:54.604568 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:54.604540 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:25:54.605110 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:25:54.604630 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:26:01.293039 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.293010 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:26:01.294661 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.293073 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:26:01.353642 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.353606 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58794c5b99-cz78h"] Jul 15 19:26:01.353895 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.353837 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58794c5b99-cz78h" podUID="17cf68f2-440c-4bd2-9d88-39e1a04aed2d" containerName="manager" containerID="cri-o://a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034" gracePeriod=30 Jul 15 19:26:01.604658 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.604635 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:26:01.613994 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.613969 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nbwg\" (UniqueName: \"kubernetes.io/projected/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-kube-api-access-7nbwg\") pod \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\" (UID: \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\") " Jul 15 19:26:01.614109 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.614011 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-webhook-cert\") pod \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\" (UID: \"17cf68f2-440c-4bd2-9d88-39e1a04aed2d\") " Jul 15 19:26:01.616303 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.616273 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "17cf68f2-440c-4bd2-9d88-39e1a04aed2d" (UID: "17cf68f2-440c-4bd2-9d88-39e1a04aed2d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:26:01.616303 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.616282 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-kube-api-access-7nbwg" (OuterVolumeSpecName: "kube-api-access-7nbwg") pod "17cf68f2-440c-4bd2-9d88-39e1a04aed2d" (UID: "17cf68f2-440c-4bd2-9d88-39e1a04aed2d"). InnerVolumeSpecName "kube-api-access-7nbwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:26:01.715037 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.714999 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nbwg\" (UniqueName: \"kubernetes.io/projected/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-kube-api-access-7nbwg\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:26:01.715037 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:01.715033 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/17cf68f2-440c-4bd2-9d88-39e1a04aed2d-webhook-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:26:02.349799 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.349761 2562 generic.go:356] "Generic (PLEG): container finished" podID="17cf68f2-440c-4bd2-9d88-39e1a04aed2d" containerID="a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034" exitCode=0 Jul 15 19:26:02.350190 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.349814 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58794c5b99-cz78h" Jul 15 19:26:02.350190 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.349811 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58794c5b99-cz78h" event={"ID":"17cf68f2-440c-4bd2-9d88-39e1a04aed2d","Type":"ContainerDied","Data":"a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034"} Jul 15 19:26:02.350190 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.349939 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58794c5b99-cz78h" event={"ID":"17cf68f2-440c-4bd2-9d88-39e1a04aed2d","Type":"ContainerDied","Data":"fa85a67a0bc6cb033d188ff153fbb24cb29b819420b4885dd79e2f6243b2463c"} Jul 15 19:26:02.350190 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.349966 2562 scope.go:117] "RemoveContainer" containerID="a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034" Jul 15 19:26:02.359105 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.358946 2562 scope.go:117] "RemoveContainer" containerID="a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034" Jul 15 19:26:02.359198 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:26:02.359182 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034\": container with ID starting with a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034 not found: ID does not exist" containerID="a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034" Jul 15 19:26:02.359246 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.359207 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034"} err="failed to get container status \"a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034\": rpc error: code = NotFound desc = could not find container \"a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034\": container with ID starting with a0798f131cff5d1e4c532a63034926e86b8a33e27cfcbd361ab39da6e363a034 not found: ID does not exist" Jul 15 19:26:02.370840 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.370814 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58794c5b99-cz78h"] Jul 15 19:26:02.375152 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.375131 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58794c5b99-cz78h"] Jul 15 19:26:02.743913 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:02.743881 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17cf68f2-440c-4bd2-9d88-39e1a04aed2d" path="/var/lib/kubelet/pods/17cf68f2-440c-4bd2-9d88-39e1a04aed2d/volumes" Jul 15 19:26:22.131463 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.131375 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf"] Jul 15 19:26:22.131901 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.131735 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17cf68f2-440c-4bd2-9d88-39e1a04aed2d" containerName="manager" Jul 15 19:26:22.131901 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.131748 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="17cf68f2-440c-4bd2-9d88-39e1a04aed2d" containerName="manager" Jul 15 19:26:22.131901 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.131804 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="17cf68f2-440c-4bd2-9d88-39e1a04aed2d" containerName="manager" Jul 15 19:26:22.137978 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.137960 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.140827 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.140807 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 15 19:26:22.140919 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.140894 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-z5qhz\"" Jul 15 19:26:22.146418 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.146392 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf"] Jul 15 19:26:22.175622 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.175589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/212d8556-45f8-4904-9343-47c9b5c99bb6-maas-api-tls\") pod \"maas-api-6d7b7f858-qsgnf\" (UID: \"212d8556-45f8-4904-9343-47c9b5c99bb6\") " pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.175827 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.175641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj94d\" (UniqueName: \"kubernetes.io/projected/212d8556-45f8-4904-9343-47c9b5c99bb6-kube-api-access-wj94d\") pod \"maas-api-6d7b7f858-qsgnf\" (UID: \"212d8556-45f8-4904-9343-47c9b5c99bb6\") " pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.175900 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.175815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/212d8556-45f8-4904-9343-47c9b5c99bb6-openshift-service-ca\") pod \"maas-api-6d7b7f858-qsgnf\" (UID: \"212d8556-45f8-4904-9343-47c9b5c99bb6\") " pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.276816 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.276787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/212d8556-45f8-4904-9343-47c9b5c99bb6-openshift-service-ca\") pod \"maas-api-6d7b7f858-qsgnf\" (UID: \"212d8556-45f8-4904-9343-47c9b5c99bb6\") " pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.277051 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.276830 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/212d8556-45f8-4904-9343-47c9b5c99bb6-maas-api-tls\") pod \"maas-api-6d7b7f858-qsgnf\" (UID: \"212d8556-45f8-4904-9343-47c9b5c99bb6\") " pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.277051 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.276851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj94d\" (UniqueName: \"kubernetes.io/projected/212d8556-45f8-4904-9343-47c9b5c99bb6-kube-api-access-wj94d\") pod \"maas-api-6d7b7f858-qsgnf\" (UID: \"212d8556-45f8-4904-9343-47c9b5c99bb6\") " pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.277613 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.277560 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/212d8556-45f8-4904-9343-47c9b5c99bb6-openshift-service-ca\") pod \"maas-api-6d7b7f858-qsgnf\" (UID: \"212d8556-45f8-4904-9343-47c9b5c99bb6\") " pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.279621 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.279598 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/212d8556-45f8-4904-9343-47c9b5c99bb6-maas-api-tls\") pod \"maas-api-6d7b7f858-qsgnf\" (UID: \"212d8556-45f8-4904-9343-47c9b5c99bb6\") " pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.286333 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.286303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj94d\" (UniqueName: \"kubernetes.io/projected/212d8556-45f8-4904-9343-47c9b5c99bb6-kube-api-access-wj94d\") pod \"maas-api-6d7b7f858-qsgnf\" (UID: \"212d8556-45f8-4904-9343-47c9b5c99bb6\") " pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.449621 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.449528 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:22.600202 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.600180 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf"] Jul 15 19:26:22.602189 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:26:22.602160 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod212d8556_45f8_4904_9343_47c9b5c99bb6.slice/crio-46a0f05aaa4e52177cd804fbae89bd90985754d670436e3c9e46668e86afb6e1 WatchSource:0}: Error finding container 46a0f05aaa4e52177cd804fbae89bd90985754d670436e3c9e46668e86afb6e1: Status 404 returned error can't find the container with id 46a0f05aaa4e52177cd804fbae89bd90985754d670436e3c9e46668e86afb6e1 Jul 15 19:26:22.603516 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.603492 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:26:22.684208 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.684176 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p"] Jul 15 19:26:22.689133 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.689105 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:22.698774 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.698748 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 19:26:22.699024 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.699010 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-q77fc\"" Jul 15 19:26:22.701691 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.701637 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p"] Jul 15 19:26:22.727632 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.727597 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-2r49c"] Jul 15 19:26:22.734292 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.734271 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:22.744885 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.744814 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-2r49c"] Jul 15 19:26:22.781119 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.781083 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d9003bf-3735-46f8-8470-cb8916dc2316-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-lcs6p\" (UID: \"7d9003bf-3735-46f8-8470-cb8916dc2316\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:22.781119 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.781118 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/072c3325-7f97-4c48-9b9d-24470a673bdb-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-2r49c\" (UID: \"072c3325-7f97-4c48-9b9d-24470a673bdb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:22.781315 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.781136 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgsgj\" (UniqueName: \"kubernetes.io/projected/072c3325-7f97-4c48-9b9d-24470a673bdb-kube-api-access-mgsgj\") pod \"payload-processing-f9b8d9b9b-2r49c\" (UID: \"072c3325-7f97-4c48-9b9d-24470a673bdb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:22.781315 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.781227 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74fxp\" (UniqueName: \"kubernetes.io/projected/7d9003bf-3735-46f8-8470-cb8916dc2316-kube-api-access-74fxp\") pod \"payload-pre-processing-74584bb9f8-lcs6p\" (UID: \"7d9003bf-3735-46f8-8470-cb8916dc2316\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:22.882366 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.882336 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74fxp\" (UniqueName: \"kubernetes.io/projected/7d9003bf-3735-46f8-8470-cb8916dc2316-kube-api-access-74fxp\") pod \"payload-pre-processing-74584bb9f8-lcs6p\" (UID: \"7d9003bf-3735-46f8-8470-cb8916dc2316\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:22.882546 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.882414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d9003bf-3735-46f8-8470-cb8916dc2316-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-lcs6p\" (UID: \"7d9003bf-3735-46f8-8470-cb8916dc2316\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:22.882546 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.882436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/072c3325-7f97-4c48-9b9d-24470a673bdb-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-2r49c\" (UID: \"072c3325-7f97-4c48-9b9d-24470a673bdb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:22.882546 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.882454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgsgj\" (UniqueName: \"kubernetes.io/projected/072c3325-7f97-4c48-9b9d-24470a673bdb-kube-api-access-mgsgj\") pod \"payload-processing-f9b8d9b9b-2r49c\" (UID: \"072c3325-7f97-4c48-9b9d-24470a673bdb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:22.883228 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.883205 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/072c3325-7f97-4c48-9b9d-24470a673bdb-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-2r49c\" (UID: \"072c3325-7f97-4c48-9b9d-24470a673bdb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:22.883312 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.883203 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d9003bf-3735-46f8-8470-cb8916dc2316-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-lcs6p\" (UID: \"7d9003bf-3735-46f8-8470-cb8916dc2316\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:22.891279 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.891255 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgsgj\" (UniqueName: \"kubernetes.io/projected/072c3325-7f97-4c48-9b9d-24470a673bdb-kube-api-access-mgsgj\") pod \"payload-processing-f9b8d9b9b-2r49c\" (UID: \"072c3325-7f97-4c48-9b9d-24470a673bdb\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:22.891399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.891364 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74fxp\" (UniqueName: \"kubernetes.io/projected/7d9003bf-3735-46f8-8470-cb8916dc2316-kube-api-access-74fxp\") pod \"payload-pre-processing-74584bb9f8-lcs6p\" (UID: \"7d9003bf-3735-46f8-8470-cb8916dc2316\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:22.999626 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:22.999526 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:23.046465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:23.046433 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:23.148276 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:23.148245 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p"] Jul 15 19:26:23.151328 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:26:23.151286 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9003bf_3735_46f8_8470_cb8916dc2316.slice/crio-b73335f534a476251bb90951c74eb374c67ade1551af42e07dc02205bd97adaa WatchSource:0}: Error finding container b73335f534a476251bb90951c74eb374c67ade1551af42e07dc02205bd97adaa: Status 404 returned error can't find the container with id b73335f534a476251bb90951c74eb374c67ade1551af42e07dc02205bd97adaa Jul 15 19:26:23.202846 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:23.202812 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-2r49c"] Jul 15 19:26:23.464745 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:23.464705 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" event={"ID":"212d8556-45f8-4904-9343-47c9b5c99bb6","Type":"ContainerStarted","Data":"46a0f05aaa4e52177cd804fbae89bd90985754d670436e3c9e46668e86afb6e1"} Jul 15 19:26:23.466294 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:23.466256 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" event={"ID":"7d9003bf-3735-46f8-8470-cb8916dc2316","Type":"ContainerStarted","Data":"b73335f534a476251bb90951c74eb374c67ade1551af42e07dc02205bd97adaa"} Jul 15 19:26:23.467416 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:23.467383 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" event={"ID":"072c3325-7f97-4c48-9b9d-24470a673bdb","Type":"ContainerStarted","Data":"049d02632e69d6ad47daac48e8aed8d251934ebedf0808f3e6ff58e7f01d2d0b"} Jul 15 19:26:25.481280 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:25.481245 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" event={"ID":"212d8556-45f8-4904-9343-47c9b5c99bb6","Type":"ContainerStarted","Data":"39afc27d9254fe8517783ae709a85c05d856d21fbe8f4e89201ed31d6b351717"} Jul 15 19:26:25.481734 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:25.481362 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:25.483127 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:25.483091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" event={"ID":"7d9003bf-3735-46f8-8470-cb8916dc2316","Type":"ContainerStarted","Data":"e8a8aee6a3108801d3172947a8e6c72c58357116a0061bd7e36779d70a317c09"} Jul 15 19:26:25.483415 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:25.483392 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:25.485734 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:25.485691 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" event={"ID":"072c3325-7f97-4c48-9b9d-24470a673bdb","Type":"ContainerStarted","Data":"332ebc39a857a48616568acdb3ceba2f9203a023993a4e4afd770f875cc7e9d3"} Jul 15 19:26:25.486205 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:25.486181 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:25.518647 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:25.518590 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" podStartSLOduration=1.052429682 podStartE2EDuration="3.518555707s" totalImagesPullingTime="2.466126025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:25.502835811 +0000 UTC m=+631.339540588" watchObservedRunningTime="2026-07-15 19:26:25.518555707 +0000 UTC m=+631.355260522" Jul 15 19:26:25.519079 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:25.519050 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" podStartSLOduration=1.655920447 podStartE2EDuration="3.519041283s" totalImagesPullingTime="1.863120836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:25.517595736 +0000 UTC m=+631.354300515" watchObservedRunningTime="2026-07-15 19:26:25.519041283 +0000 UTC m=+631.355746059" Jul 15 19:26:25.536896 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:25.536852 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" podStartSLOduration=1.61872799 podStartE2EDuration="3.536832758s" totalImagesPullingTime="1.918104768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:25.536161424 +0000 UTC m=+631.372866200" watchObservedRunningTime="2026-07-15 19:26:25.536832758 +0000 UTC m=+631.373537539" Jul 15 19:26:30.518763 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:30.518732 2562 generic.go:356] "Generic (PLEG): container finished" podID="9de8a364-dd9e-47bb-9520-3a51688ff72f" containerID="b5dc0cb270a6477c4ce4a016e6c9074cbebe41d576a94e0701fb5a9c4ca12607" exitCode=0 Jul 15 19:26:30.519153 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:30.518803 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" event={"ID":"9de8a364-dd9e-47bb-9520-3a51688ff72f","Type":"ContainerDied","Data":"b5dc0cb270a6477c4ce4a016e6c9074cbebe41d576a94e0701fb5a9c4ca12607"} Jul 15 19:26:30.519193 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:30.519182 2562 scope.go:117] "RemoveContainer" containerID="b5dc0cb270a6477c4ce4a016e6c9074cbebe41d576a94e0701fb5a9c4ca12607" Jul 15 19:26:30.520100 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:30.520074 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:26:30.520153 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:30.520140 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:26:30.520193 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:30.520173 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:26:31.506970 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:31.506895 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d7b7f858-qsgnf" Jul 15 19:26:31.527202 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:31.527170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" event={"ID":"9de8a364-dd9e-47bb-9520-3a51688ff72f","Type":"ContainerStarted","Data":"480e825ae3bb737569db305efedbbf3692d2b2a8b37b9ea9247315490e512d7a"} Jul 15 19:26:31.783547 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:31.783443 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:26:31.784861 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:31.784825 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ts9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 15 19:26:31.784991 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:31.784893 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" podUID="9de8a364-dd9e-47bb-9520-3a51688ff72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 15 19:26:32.784415 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:32.784375 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ts9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 15 19:26:32.785021 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:32.784991 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" podUID="9de8a364-dd9e-47bb-9520-3a51688ff72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 15 19:26:33.783550 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:33.783502 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ts9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 15 19:26:33.783784 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:33.783605 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" podUID="9de8a364-dd9e-47bb-9520-3a51688ff72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 15 19:26:34.798069 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:34.798025 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ts9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 15 19:26:34.798561 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:34.798115 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" podUID="9de8a364-dd9e-47bb-9520-3a51688ff72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 15 19:26:35.003776 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:35.003732 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:26:35.787768 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:35.787738 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:26:35.788746 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:35.788726 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ts9ph" Jul 15 19:26:37.502796 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:37.502759 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-2r49c" Jul 15 19:26:37.503175 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:37.503007 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-lcs6p" Jul 15 19:26:48.835057 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.835013 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq"] Jul 15 19:26:48.838070 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.838047 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:48.842474 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.842452 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 19:26:48.842630 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.842452 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dx2cn\"" Jul 15 19:26:48.842630 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.842459 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 15 19:26:48.842630 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.842500 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 19:26:48.853477 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.853443 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq"] Jul 15 19:26:48.944908 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.944871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:48.945076 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.944945 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:48.945076 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.944989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2jvt\" (UniqueName: \"kubernetes.io/projected/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-kube-api-access-f2jvt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:48.945076 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.945011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:48.945173 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.945104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:48.945209 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:48.945172 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.046203 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.046161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.046388 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.046224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2jvt\" (UniqueName: \"kubernetes.io/projected/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-kube-api-access-f2jvt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.046388 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.046242 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.046388 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.046260 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.046388 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.046295 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.046388 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.046313 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.046699 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.046677 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.046699 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.046689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.046800 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.046779 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.048781 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.048757 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.049013 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.048991 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.054820 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.054796 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2jvt\" (UniqueName: \"kubernetes.io/projected/a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312-kube-api-access-f2jvt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xsxkq\" (UID: \"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.150367 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.150327 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:49.292310 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.292281 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq"] Jul 15 19:26:49.293862 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:26:49.293831 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda17cfcb5_4e21_4d2b_8e9e_757a4fb9d312.slice/crio-633a5e31caabfed934089c9560ea920602e0f90287febe7d2ea69bfb07d40f05 WatchSource:0}: Error finding container 633a5e31caabfed934089c9560ea920602e0f90287febe7d2ea69bfb07d40f05: Status 404 returned error can't find the container with id 633a5e31caabfed934089c9560ea920602e0f90287febe7d2ea69bfb07d40f05 Jul 15 19:26:49.628238 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:49.628114 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" event={"ID":"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312","Type":"ContainerStarted","Data":"633a5e31caabfed934089c9560ea920602e0f90287febe7d2ea69bfb07d40f05"} Jul 15 19:26:50.635735 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:50.635689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" event={"ID":"a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312","Type":"ContainerStarted","Data":"133f3bc295f30d1fa2f02de4918e9cc99a9a28bf371bda10f6b98d02ed35a6eb"} Jul 15 19:26:50.654551 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:50.654470 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" podStartSLOduration=1.634276356 podStartE2EDuration="2.654455783s" totalImagesPullingTime="1.020179427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:50.653261169 +0000 UTC m=+656.489965961" watchObservedRunningTime="2026-07-15 19:26:50.654455783 +0000 UTC m=+656.491160558" Jul 15 19:26:59.151369 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:59.151333 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:59.151830 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:59.151413 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:59.163276 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:59.163252 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:26:59.692082 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:26:59.692052 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xsxkq" Jul 15 19:27:15.851819 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.851780 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8"] Jul 15 19:27:15.856926 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.856909 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:15.859661 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.859640 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 15 19:27:15.864932 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.864911 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8"] Jul 15 19:27:15.983817 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.983783 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:15.983817 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.983827 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9jfb\" (UniqueName: \"kubernetes.io/projected/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-kube-api-access-h9jfb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:15.984056 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.983942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:15.984056 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.983974 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:15.984056 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.983996 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:15.984056 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:15.984050 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.084619 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.084553 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9jfb\" (UniqueName: \"kubernetes.io/projected/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-kube-api-access-h9jfb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.084824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.084676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.084824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.084707 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.084824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.084731 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.084824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.084765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.084824 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.084796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.085128 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.085102 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.085128 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.085120 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.085233 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.085158 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.087222 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.087197 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.087511 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.087491 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.092807 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.092785 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9jfb\" (UniqueName: \"kubernetes.io/projected/ff95dfdc-6555-4409-8f2c-ffe7f73938c2-kube-api-access-h9jfb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8\" (UID: \"ff95dfdc-6555-4409-8f2c-ffe7f73938c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.168703 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.168668 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:16.300729 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.300704 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8"] Jul 15 19:27:16.302249 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:27:16.302219 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff95dfdc_6555_4409_8f2c_ffe7f73938c2.slice/crio-69f436258a21b9ea1d12d911975cc143f9f1dfae7bc1d33d26ee7f1504447d0e WatchSource:0}: Error finding container 69f436258a21b9ea1d12d911975cc143f9f1dfae7bc1d33d26ee7f1504447d0e: Status 404 returned error can't find the container with id 69f436258a21b9ea1d12d911975cc143f9f1dfae7bc1d33d26ee7f1504447d0e Jul 15 19:27:16.768970 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.768934 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" event={"ID":"ff95dfdc-6555-4409-8f2c-ffe7f73938c2","Type":"ContainerStarted","Data":"814656898efe7cb124f38dd5b48187d56dcb5ccc62d6231d2c5e70f3640021f3"} Jul 15 19:27:16.769166 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.768994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" event={"ID":"ff95dfdc-6555-4409-8f2c-ffe7f73938c2","Type":"ContainerStarted","Data":"69f436258a21b9ea1d12d911975cc143f9f1dfae7bc1d33d26ee7f1504447d0e"} Jul 15 19:27:16.788543 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:16.788487 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" podStartSLOduration=1.548616432 podStartE2EDuration="1.788470247s" totalImagesPullingTime="239.853815ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:27:16.786312535 +0000 UTC m=+682.623017312" watchObservedRunningTime="2026-07-15 19:27:16.788470247 +0000 UTC m=+682.625175024" Jul 15 19:27:26.169319 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:26.169266 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:26.169319 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:26.169338 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:26.181506 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:26.181485 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:26.831652 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:26.831623 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8" Jul 15 19:27:31.317895 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.317862 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42"] Jul 15 19:27:31.323698 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.323680 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.326506 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.326490 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 15 19:27:31.332858 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.332829 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42"] Jul 15 19:27:31.429423 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.429381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/219db70a-33ce-4fbe-885b-7491928fdcc1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.429615 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.429446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.429615 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.429475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.429615 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.429492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8xj6\" (UniqueName: \"kubernetes.io/projected/219db70a-33ce-4fbe-885b-7491928fdcc1-kube-api-access-v8xj6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.429721 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.429618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.429721 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.429639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.530282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.530240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.530282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.530279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.530490 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.530305 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8xj6\" (UniqueName: \"kubernetes.io/projected/219db70a-33ce-4fbe-885b-7491928fdcc1-kube-api-access-v8xj6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.530490 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.530354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.530490 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.530375 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.530490 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.530413 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/219db70a-33ce-4fbe-885b-7491928fdcc1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.530821 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.530796 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.530889 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.530816 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.531021 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.531004 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.532998 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.532968 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/219db70a-33ce-4fbe-885b-7491928fdcc1-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.533148 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.533131 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/219db70a-33ce-4fbe-885b-7491928fdcc1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.538891 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.538865 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8xj6\" (UniqueName: \"kubernetes.io/projected/219db70a-33ce-4fbe-885b-7491928fdcc1-kube-api-access-v8xj6\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42\" (UID: \"219db70a-33ce-4fbe-885b-7491928fdcc1\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.633632 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.633601 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:31.773948 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.773921 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42"] Jul 15 19:27:31.776903 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:27:31.776875 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod219db70a_33ce_4fbe_885b_7491928fdcc1.slice/crio-336af428291526937c98d82e9ce98e2369a5f0e168c79f0cee596d40ab9ce521 WatchSource:0}: Error finding container 336af428291526937c98d82e9ce98e2369a5f0e168c79f0cee596d40ab9ce521: Status 404 returned error can't find the container with id 336af428291526937c98d82e9ce98e2369a5f0e168c79f0cee596d40ab9ce521 Jul 15 19:27:31.848203 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:31.848164 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" event={"ID":"219db70a-33ce-4fbe-885b-7491928fdcc1","Type":"ContainerStarted","Data":"336af428291526937c98d82e9ce98e2369a5f0e168c79f0cee596d40ab9ce521"} Jul 15 19:27:32.854658 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:32.854617 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" event={"ID":"219db70a-33ce-4fbe-885b-7491928fdcc1","Type":"ContainerStarted","Data":"d3208d47f1a8da3bac9546903933c93731debcfdf4b0e0dba34934f6e407b4c7"} Jul 15 19:27:32.873186 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:32.873137 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" podStartSLOduration=1.669872488 podStartE2EDuration="1.873121182s" totalImagesPullingTime="203.248694ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:27:32.871262456 +0000 UTC m=+698.707967232" watchObservedRunningTime="2026-07-15 19:27:32.873121182 +0000 UTC m=+698.709825958" Jul 15 19:27:41.634632 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:41.634568 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:41.634632 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:41.634640 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:41.646718 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:41.646690 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:27:41.914028 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:27:41.913938 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42" Jul 15 19:28:48.364610 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.364562 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-548f6cb7d-8f7s6"] Jul 15 19:28:48.367899 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.367882 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548f6cb7d-8f7s6" Jul 15 19:28:48.372180 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.372153 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 15 19:28:48.372443 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.372420 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-r9c97\"" Jul 15 19:28:48.375957 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.375935 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-548f6cb7d-8f7s6"] Jul 15 19:28:48.504807 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.504776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nmsr\" (UniqueName: \"kubernetes.io/projected/ca4c8d19-3217-4017-99c7-8bdaf5d4ca38-kube-api-access-2nmsr\") pod \"authorino-548f6cb7d-8f7s6\" (UID: \"ca4c8d19-3217-4017-99c7-8bdaf5d4ca38\") " pod="kuadrant-system/authorino-548f6cb7d-8f7s6" Jul 15 19:28:48.504980 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.504893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca4c8d19-3217-4017-99c7-8bdaf5d4ca38-tls-cert\") pod \"authorino-548f6cb7d-8f7s6\" (UID: \"ca4c8d19-3217-4017-99c7-8bdaf5d4ca38\") " pod="kuadrant-system/authorino-548f6cb7d-8f7s6" Jul 15 19:28:48.605530 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.605492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca4c8d19-3217-4017-99c7-8bdaf5d4ca38-tls-cert\") pod \"authorino-548f6cb7d-8f7s6\" (UID: \"ca4c8d19-3217-4017-99c7-8bdaf5d4ca38\") " pod="kuadrant-system/authorino-548f6cb7d-8f7s6" Jul 15 19:28:48.605758 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.605542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nmsr\" (UniqueName: \"kubernetes.io/projected/ca4c8d19-3217-4017-99c7-8bdaf5d4ca38-kube-api-access-2nmsr\") pod \"authorino-548f6cb7d-8f7s6\" (UID: \"ca4c8d19-3217-4017-99c7-8bdaf5d4ca38\") " pod="kuadrant-system/authorino-548f6cb7d-8f7s6" Jul 15 19:28:48.608108 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.608084 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca4c8d19-3217-4017-99c7-8bdaf5d4ca38-tls-cert\") pod \"authorino-548f6cb7d-8f7s6\" (UID: \"ca4c8d19-3217-4017-99c7-8bdaf5d4ca38\") " pod="kuadrant-system/authorino-548f6cb7d-8f7s6" Jul 15 19:28:48.613859 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.613834 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nmsr\" (UniqueName: \"kubernetes.io/projected/ca4c8d19-3217-4017-99c7-8bdaf5d4ca38-kube-api-access-2nmsr\") pod \"authorino-548f6cb7d-8f7s6\" (UID: \"ca4c8d19-3217-4017-99c7-8bdaf5d4ca38\") " pod="kuadrant-system/authorino-548f6cb7d-8f7s6" Jul 15 19:28:48.678422 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.678392 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-548f6cb7d-8f7s6" Jul 15 19:28:48.820106 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:48.820075 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-548f6cb7d-8f7s6"] Jul 15 19:28:48.821194 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:28:48.821167 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca4c8d19_3217_4017_99c7_8bdaf5d4ca38.slice/crio-a3ecb76164c4b3a3d8c0f2e3acf35703035416804aa349aea987788f46de9982 WatchSource:0}: Error finding container a3ecb76164c4b3a3d8c0f2e3acf35703035416804aa349aea987788f46de9982: Status 404 returned error can't find the container with id a3ecb76164c4b3a3d8c0f2e3acf35703035416804aa349aea987788f46de9982 Jul 15 19:28:49.263878 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:49.263840 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548f6cb7d-8f7s6" event={"ID":"ca4c8d19-3217-4017-99c7-8bdaf5d4ca38","Type":"ContainerStarted","Data":"a3ecb76164c4b3a3d8c0f2e3acf35703035416804aa349aea987788f46de9982"} Jul 15 19:28:51.276406 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:51.276369 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-548f6cb7d-8f7s6" event={"ID":"ca4c8d19-3217-4017-99c7-8bdaf5d4ca38","Type":"ContainerStarted","Data":"c48e4aec3b4846c9f7c8a0db1e61a383c67bca01d6420db23749442b2c633a80"} Jul 15 19:28:51.295805 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:28:51.295638 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-548f6cb7d-8f7s6" podStartSLOduration=1.853749978 podStartE2EDuration="3.295622356s" totalImagesPullingTime="1.441872378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:28:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:51.294164231 +0000 UTC m=+777.130869010" watchObservedRunningTime="2026-07-15 19:28:51.295622356 +0000 UTC m=+777.132327133" Jul 15 19:30:00.865792 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:00.865758 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cb8df54d8-j6lwc"] Jul 15 19:30:00.866396 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:00.865973 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" podUID="b8ed2272-962a-4cfa-b843-a40349dc4393" containerName="manager" containerID="cri-o://bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a" gracePeriod=30 Jul 15 19:30:01.124267 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.124210 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:30:01.270758 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.270727 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8ed2272-962a-4cfa-b843-a40349dc4393-webhook-cert\") pod \"b8ed2272-962a-4cfa-b843-a40349dc4393\" (UID: \"b8ed2272-962a-4cfa-b843-a40349dc4393\") " Jul 15 19:30:01.270933 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.270788 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rs6nm\" (UniqueName: \"kubernetes.io/projected/b8ed2272-962a-4cfa-b843-a40349dc4393-kube-api-access-rs6nm\") pod \"b8ed2272-962a-4cfa-b843-a40349dc4393\" (UID: \"b8ed2272-962a-4cfa-b843-a40349dc4393\") " Jul 15 19:30:01.272957 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.272933 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ed2272-962a-4cfa-b843-a40349dc4393-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b8ed2272-962a-4cfa-b843-a40349dc4393" (UID: "b8ed2272-962a-4cfa-b843-a40349dc4393"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:30:01.273114 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.273097 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ed2272-962a-4cfa-b843-a40349dc4393-kube-api-access-rs6nm" (OuterVolumeSpecName: "kube-api-access-rs6nm") pod "b8ed2272-962a-4cfa-b843-a40349dc4393" (UID: "b8ed2272-962a-4cfa-b843-a40349dc4393"). InnerVolumeSpecName "kube-api-access-rs6nm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:30:01.371851 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.371809 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs6nm\" (UniqueName: \"kubernetes.io/projected/b8ed2272-962a-4cfa-b843-a40349dc4393-kube-api-access-rs6nm\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:30:01.371851 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.371844 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b8ed2272-962a-4cfa-b843-a40349dc4393-webhook-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:30:01.664966 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.664927 2562 generic.go:356] "Generic (PLEG): container finished" podID="b8ed2272-962a-4cfa-b843-a40349dc4393" containerID="bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a" exitCode=0 Jul 15 19:30:01.665143 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.664972 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" Jul 15 19:30:01.665143 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.665011 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" event={"ID":"b8ed2272-962a-4cfa-b843-a40349dc4393","Type":"ContainerDied","Data":"bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a"} Jul 15 19:30:01.665143 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.665062 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb8df54d8-j6lwc" event={"ID":"b8ed2272-962a-4cfa-b843-a40349dc4393","Type":"ContainerDied","Data":"d4f6114a079e3207e17768ad071281c20ef48e3902467c0f2e92dc8b71dce3d7"} Jul 15 19:30:01.665143 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.665085 2562 scope.go:117] "RemoveContainer" containerID="bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a" Jul 15 19:30:01.674592 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.674560 2562 scope.go:117] "RemoveContainer" containerID="bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a" Jul 15 19:30:01.674875 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:30:01.674858 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a\": container with ID starting with bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a not found: ID does not exist" containerID="bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a" Jul 15 19:30:01.674920 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.674884 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a"} err="failed to get container status \"bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a\": rpc error: code = NotFound desc = could not find container \"bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a\": container with ID starting with bd10962ffd71320bb404dd90e9341ec30c95bd9754a272ad48c38fe4f47e552a not found: ID does not exist" Jul 15 19:30:01.686508 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.686480 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cb8df54d8-j6lwc"] Jul 15 19:30:01.692228 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:01.692203 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cb8df54d8-j6lwc"] Jul 15 19:30:02.530595 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.530550 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cb8df54d8-dzdnd"] Jul 15 19:30:02.530950 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.530939 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8ed2272-962a-4cfa-b843-a40349dc4393" containerName="manager" Jul 15 19:30:02.531119 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.530951 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8ed2272-962a-4cfa-b843-a40349dc4393" containerName="manager" Jul 15 19:30:02.531119 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.531010 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8ed2272-962a-4cfa-b843-a40349dc4393" containerName="manager" Jul 15 19:30:02.535394 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.535373 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:02.537929 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.537908 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 19:30:02.539264 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.539241 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 19:30:02.539346 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.539278 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2d9mb\"" Jul 15 19:30:02.544357 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.544335 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb8df54d8-dzdnd"] Jul 15 19:30:02.578241 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.578206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d09d6a2b-890d-42a4-a0bf-47b61fb821cf-webhook-cert\") pod \"maas-controller-6cb8df54d8-dzdnd\" (UID: \"d09d6a2b-890d-42a4-a0bf-47b61fb821cf\") " pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:02.578417 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.578251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdqj5\" (UniqueName: \"kubernetes.io/projected/d09d6a2b-890d-42a4-a0bf-47b61fb821cf-kube-api-access-xdqj5\") pod \"maas-controller-6cb8df54d8-dzdnd\" (UID: \"d09d6a2b-890d-42a4-a0bf-47b61fb821cf\") " pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:02.678934 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.678901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d09d6a2b-890d-42a4-a0bf-47b61fb821cf-webhook-cert\") pod \"maas-controller-6cb8df54d8-dzdnd\" (UID: \"d09d6a2b-890d-42a4-a0bf-47b61fb821cf\") " pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:02.679109 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.678953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqj5\" (UniqueName: \"kubernetes.io/projected/d09d6a2b-890d-42a4-a0bf-47b61fb821cf-kube-api-access-xdqj5\") pod \"maas-controller-6cb8df54d8-dzdnd\" (UID: \"d09d6a2b-890d-42a4-a0bf-47b61fb821cf\") " pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:02.681419 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.681388 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d09d6a2b-890d-42a4-a0bf-47b61fb821cf-webhook-cert\") pod \"maas-controller-6cb8df54d8-dzdnd\" (UID: \"d09d6a2b-890d-42a4-a0bf-47b61fb821cf\") " pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:02.687474 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.687451 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdqj5\" (UniqueName: \"kubernetes.io/projected/d09d6a2b-890d-42a4-a0bf-47b61fb821cf-kube-api-access-xdqj5\") pod \"maas-controller-6cb8df54d8-dzdnd\" (UID: \"d09d6a2b-890d-42a4-a0bf-47b61fb821cf\") " pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:02.744113 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.744080 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8ed2272-962a-4cfa-b843-a40349dc4393" path="/var/lib/kubelet/pods/b8ed2272-962a-4cfa-b843-a40349dc4393/volumes" Jul 15 19:30:02.846994 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.846904 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:02.978550 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:02.978516 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cb8df54d8-dzdnd"] Jul 15 19:30:02.980254 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:30:02.980227 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09d6a2b_890d_42a4_a0bf_47b61fb821cf.slice/crio-b77a0f603c3a18caff668a8910af88c10cf6d28e8e430a6c9096c6e465a9c89b WatchSource:0}: Error finding container b77a0f603c3a18caff668a8910af88c10cf6d28e8e430a6c9096c6e465a9c89b: Status 404 returned error can't find the container with id b77a0f603c3a18caff668a8910af88c10cf6d28e8e430a6c9096c6e465a9c89b Jul 15 19:30:03.678375 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:03.678336 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" event={"ID":"d09d6a2b-890d-42a4-a0bf-47b61fb821cf","Type":"ContainerStarted","Data":"b77a0f603c3a18caff668a8910af88c10cf6d28e8e430a6c9096c6e465a9c89b"} Jul 15 19:30:05.691239 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:05.691203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" event={"ID":"d09d6a2b-890d-42a4-a0bf-47b61fb821cf","Type":"ContainerStarted","Data":"85f904350b0c64db2fd80897a26e16493b2c97f5e41fe04fed63c47aacb3f234"} Jul 15 19:30:05.691703 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:05.691318 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:05.710476 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:05.710429 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" podStartSLOduration=1.926438842 podStartE2EDuration="3.710415033s" totalImagesPullingTime="1.783976191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:05.70975459 +0000 UTC m=+851.546459370" watchObservedRunningTime="2026-07-15 19:30:05.710415033 +0000 UTC m=+851.547119810" Jul 15 19:30:16.699326 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:16.699297 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cb8df54d8-dzdnd" Jul 15 19:30:54.642641 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:54.642542 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:30:54.645599 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:30:54.645548 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:35:54.680190 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:35:54.680157 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:35:54.684348 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:35:54.684327 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:40:54.715010 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:40:54.714974 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:40:54.719434 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:40:54.719411 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:42:04.049068 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.049027 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7"] Jul 15 19:42:04.052596 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.052552 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.055407 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.055380 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-daf69041-openshift-default-dockercfg-8vkr9\"" Jul 15 19:42:04.075645 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.075612 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7"] Jul 15 19:42:04.206999 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.206951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istiod-ca-cert\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.206999 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.207008 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-podinfo\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.207235 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.207038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-data\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.207235 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.207089 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-socket\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.207235 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.207115 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-token\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.207235 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.207135 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-certs\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.207235 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.207165 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlmnf\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-kube-api-access-wlmnf\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.207416 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.207246 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-envoy\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.207416 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.207268 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-credential-socket\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308175 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-envoy\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308233 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-credential-socket\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istiod-ca-cert\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308557 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308285 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-podinfo\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308557 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-data\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308557 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-socket\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308557 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-token\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308557 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308399 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-certs\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308557 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308430 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlmnf\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-kube-api-access-wlmnf\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308862 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308685 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-credential-socket\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308862 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308727 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-data\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.308862 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308821 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-socket\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.309021 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.308969 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-certs\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.309139 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.309116 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istiod-ca-cert\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.310564 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.310545 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-envoy\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.310986 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.310970 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-podinfo\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.317480 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.317457 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-token\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.317677 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.317659 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlmnf\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-kube-api-access-wlmnf\") pod \"e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.362783 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.362750 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:04.503045 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.503018 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7"] Jul 15 19:42:04.505890 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:42:04.505861 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d91014_e88b_4e2a_8082_fb16dd4fde8d.slice/crio-f25644e26692ce3d6e80fb9e86a807a0da97e46d285f69dff7f2c83d520bc3c1 WatchSource:0}: Error finding container f25644e26692ce3d6e80fb9e86a807a0da97e46d285f69dff7f2c83d520bc3c1: Status 404 returned error can't find the container with id f25644e26692ce3d6e80fb9e86a807a0da97e46d285f69dff7f2c83d520bc3c1 Jul 15 19:42:04.508352 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.508318 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:42:04.508431 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.508387 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:42:04.508431 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.508417 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:42:04.670260 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.670224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" event={"ID":"85d91014-e88b-4e2a-8082-fb16dd4fde8d","Type":"ContainerStarted","Data":"3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796"} Jul 15 19:42:04.670431 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.670285 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" event={"ID":"85d91014-e88b-4e2a-8082-fb16dd4fde8d","Type":"ContainerStarted","Data":"f25644e26692ce3d6e80fb9e86a807a0da97e46d285f69dff7f2c83d520bc3c1"} Jul 15 19:42:04.692729 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:04.692676 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" podStartSLOduration=0.692558263 podStartE2EDuration="692.558263ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:42:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:42:04.691514303 +0000 UTC m=+1570.528219092" watchObservedRunningTime="2026-07-15 19:42:04.692558263 +0000 UTC m=+1570.529263039" Jul 15 19:42:05.363078 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:05.363038 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:05.391733 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:05.391705 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:05.676120 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:05.676087 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:05.677244 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:05.677222 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:16.744698 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:16.744662 2562 generic.go:356] "Generic (PLEG): container finished" podID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerID="3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796" exitCode=0 Jul 15 19:42:16.745098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:16.744730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" event={"ID":"85d91014-e88b-4e2a-8082-fb16dd4fde8d","Type":"ContainerDied","Data":"3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796"} Jul 15 19:42:16.745144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:16.745117 2562 scope.go:117] "RemoveContainer" containerID="3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796" Jul 15 19:42:16.746086 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:16.746060 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:42:16.746141 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:16.746127 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:42:16.746179 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:16.746156 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:42:17.752450 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:17.752413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" event={"ID":"85d91014-e88b-4e2a-8082-fb16dd4fde8d","Type":"ContainerStarted","Data":"e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237"} Jul 15 19:42:18.363652 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:18.363612 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:18.364670 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:18.364643 2562 patch_prober.go:28] interesting pod/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 15 19:42:18.364802 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:18.364693 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 15 19:42:19.363413 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:19.363381 2562 patch_prober.go:28] interesting pod/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 15 19:42:19.363807 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:19.363445 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 15 19:42:20.363260 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:20.363225 2562 patch_prober.go:28] interesting pod/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 15 19:42:20.363441 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:20.363278 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 15 19:42:20.676383 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:20.676335 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:21.363427 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:21.363390 2562 patch_prober.go:28] interesting pod/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 15 19:42:21.363627 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:21.363445 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 15 19:42:22.367127 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:22.367094 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:42:22.368177 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:42:22.368157 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:43:15.725347 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:15.725306 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7"] Jul 15 19:43:15.725849 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:15.725549 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" containerID="cri-o://e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237" gracePeriod=30 Jul 15 19:43:20.985881 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:20.985812 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:43:21.100257 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.100224 2562 generic.go:356] "Generic (PLEG): container finished" podID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerID="e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237" exitCode=0 Jul 15 19:43:21.100427 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.100273 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" Jul 15 19:43:21.100427 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.100318 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" event={"ID":"85d91014-e88b-4e2a-8082-fb16dd4fde8d","Type":"ContainerDied","Data":"e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237"} Jul 15 19:43:21.100427 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.100374 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7" event={"ID":"85d91014-e88b-4e2a-8082-fb16dd4fde8d","Type":"ContainerDied","Data":"f25644e26692ce3d6e80fb9e86a807a0da97e46d285f69dff7f2c83d520bc3c1"} Jul 15 19:43:21.100427 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.100400 2562 scope.go:117] "RemoveContainer" containerID="e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237" Jul 15 19:43:21.109986 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.109971 2562 scope.go:117] "RemoveContainer" containerID="3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796" Jul 15 19:43:21.119308 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.119285 2562 scope.go:117] "RemoveContainer" containerID="e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237" Jul 15 19:43:21.119671 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:43:21.119652 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237\": container with ID starting with e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237 not found: ID does not exist" containerID="e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237" Jul 15 19:43:21.119755 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.119682 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237"} err="failed to get container status \"e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237\": rpc error: code = NotFound desc = could not find container \"e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237\": container with ID starting with e38e4903f1a87a20a82cb3cd8aa14831e06aab04d237de342fb33cc485928237 not found: ID does not exist" Jul 15 19:43:21.119755 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.119700 2562 scope.go:117] "RemoveContainer" containerID="3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796" Jul 15 19:43:21.119954 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:43:21.119935 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796\": container with ID starting with 3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796 not found: ID does not exist" containerID="3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796" Jul 15 19:43:21.119996 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.119959 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796"} err="failed to get container status \"3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796\": rpc error: code = NotFound desc = could not find container \"3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796\": container with ID starting with 3963407131b8a1166f643884d8ee5e69261034fd7cacdd62888f4450a4213796 not found: ID does not exist" Jul 15 19:43:21.133302 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133270 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-envoy\") pod \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " Jul 15 19:43:21.133302 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133306 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-socket\") pod \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " Jul 15 19:43:21.133437 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133363 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-token\") pod \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " Jul 15 19:43:21.133472 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133436 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istiod-ca-cert\") pod \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " Jul 15 19:43:21.133519 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133469 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-credential-socket\") pod \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " Jul 15 19:43:21.133519 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133506 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-podinfo\") pod \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " Jul 15 19:43:21.133634 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133556 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-certs\") pod \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " Jul 15 19:43:21.133634 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133616 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlmnf\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-kube-api-access-wlmnf\") pod \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " Jul 15 19:43:21.133724 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133645 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-data\") pod \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\" (UID: \"85d91014-e88b-4e2a-8082-fb16dd4fde8d\") " Jul 15 19:43:21.133724 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133663 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85d91014-e88b-4e2a-8082-fb16dd4fde8d" (UID: "85d91014-e88b-4e2a-8082-fb16dd4fde8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:43:21.133848 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133817 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85d91014-e88b-4e2a-8082-fb16dd4fde8d" (UID: "85d91014-e88b-4e2a-8082-fb16dd4fde8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:43:21.133903 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133853 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85d91014-e88b-4e2a-8082-fb16dd4fde8d" (UID: "85d91014-e88b-4e2a-8082-fb16dd4fde8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:43:21.133993 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.133945 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85d91014-e88b-4e2a-8082-fb16dd4fde8d" (UID: "85d91014-e88b-4e2a-8082-fb16dd4fde8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:43:21.134070 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.134046 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-socket\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:43:21.134070 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.134073 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istiod-ca-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:43:21.134259 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.134091 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-credential-socket\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:43:21.134472 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.134445 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-data" (OuterVolumeSpecName: "istio-data") pod "85d91014-e88b-4e2a-8082-fb16dd4fde8d" (UID: "85d91014-e88b-4e2a-8082-fb16dd4fde8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:43:21.136074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.136042 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85d91014-e88b-4e2a-8082-fb16dd4fde8d" (UID: "85d91014-e88b-4e2a-8082-fb16dd4fde8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:43:21.136177 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.136092 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-kube-api-access-wlmnf" (OuterVolumeSpecName: "kube-api-access-wlmnf") pod "85d91014-e88b-4e2a-8082-fb16dd4fde8d" (UID: "85d91014-e88b-4e2a-8082-fb16dd4fde8d"). InnerVolumeSpecName "kube-api-access-wlmnf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:43:21.136177 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.136164 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85d91014-e88b-4e2a-8082-fb16dd4fde8d" (UID: "85d91014-e88b-4e2a-8082-fb16dd4fde8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 19:43:21.136271 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.136179 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-token" (OuterVolumeSpecName: "istio-token") pod "85d91014-e88b-4e2a-8082-fb16dd4fde8d" (UID: "85d91014-e88b-4e2a-8082-fb16dd4fde8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:43:21.234869 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.234829 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-podinfo\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:43:21.234869 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.234862 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-workload-certs\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:43:21.234869 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.234875 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlmnf\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-kube-api-access-wlmnf\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:43:21.235110 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.234889 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-data\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:43:21.235110 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.234900 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-envoy\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:43:21.235110 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.234910 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85d91014-e88b-4e2a-8082-fb16dd4fde8d-istio-token\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:43:21.422684 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.422638 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7"] Jul 15 19:43:21.426534 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:21.426502 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-daf69041-openshift-default-567f9bb7df-7ncq7"] Jul 15 19:43:22.118092 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.118059 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6"] Jul 15 19:43:22.118559 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.118480 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" Jul 15 19:43:22.118559 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.118493 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" Jul 15 19:43:22.118559 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.118502 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" Jul 15 19:43:22.118559 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.118508 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" Jul 15 19:43:22.118807 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.118608 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" Jul 15 19:43:22.118807 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.118620 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" containerName="istio-proxy" Jul 15 19:43:22.123223 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.123203 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.126465 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.126440 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5827a521-gw-openshift-default-dockercfg-hzdgq\"" Jul 15 19:43:22.135304 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.135270 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6"] Jul 15 19:43:22.243308 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.243265 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-data\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.243546 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.243328 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-certs\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.243546 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.243433 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lds2l\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-kube-api-access-lds2l\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.243546 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.243489 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-credential-socket\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.243731 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.243542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/474e60d7-43b1-46a3-9459-4bfe94d722bb-istiod-ca-cert\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.243731 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.243599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-envoy\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.243731 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.243634 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-token\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.243731 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.243668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-socket\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.243731 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.243693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-podinfo\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.344538 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344479 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-socket\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.344538 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344538 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-podinfo\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.344834 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-data\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.344834 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344652 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-certs\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.344834 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lds2l\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-kube-api-access-lds2l\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.344834 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-credential-socket\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.344834 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/474e60d7-43b1-46a3-9459-4bfe94d722bb-istiod-ca-cert\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.344834 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344811 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-envoy\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.345131 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344854 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-token\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.345131 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.344923 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-socket\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.345131 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.345072 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-data\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.345491 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.345283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-certs\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.345491 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.345401 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-credential-socket\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.345615 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.345511 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/474e60d7-43b1-46a3-9459-4bfe94d722bb-istiod-ca-cert\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.347165 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.347141 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-envoy\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.347283 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.347168 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-podinfo\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.352628 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.352605 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-token\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.352807 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.352788 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lds2l\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-kube-api-access-lds2l\") pod \"e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.435956 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.435917 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:22.585060 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.584957 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6"] Jul 15 19:43:22.588203 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:43:22.588173 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod474e60d7_43b1_46a3_9459_4bfe94d722bb.slice/crio-04018896ace2cae5aed451bc5267c2bed0a5c4ffaf836ddedef6bdada5c1e08d WatchSource:0}: Error finding container 04018896ace2cae5aed451bc5267c2bed0a5c4ffaf836ddedef6bdada5c1e08d: Status 404 returned error can't find the container with id 04018896ace2cae5aed451bc5267c2bed0a5c4ffaf836ddedef6bdada5c1e08d Jul 15 19:43:22.590680 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.590555 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:43:22.590680 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.590670 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:43:22.590680 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.590710 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:43:22.747224 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:22.747136 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85d91014-e88b-4e2a-8082-fb16dd4fde8d" path="/var/lib/kubelet/pods/85d91014-e88b-4e2a-8082-fb16dd4fde8d/volumes" Jul 15 19:43:23.121627 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:23.121509 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" event={"ID":"474e60d7-43b1-46a3-9459-4bfe94d722bb","Type":"ContainerStarted","Data":"c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91"} Jul 15 19:43:23.121627 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:23.121561 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" event={"ID":"474e60d7-43b1-46a3-9459-4bfe94d722bb","Type":"ContainerStarted","Data":"04018896ace2cae5aed451bc5267c2bed0a5c4ffaf836ddedef6bdada5c1e08d"} Jul 15 19:43:23.142840 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:23.142788 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" podStartSLOduration=1.142770103 podStartE2EDuration="1.142770103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:43:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:43:23.141258099 +0000 UTC m=+1648.977962873" watchObservedRunningTime="2026-07-15 19:43:23.142770103 +0000 UTC m=+1648.979474880" Jul 15 19:43:23.436764 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:23.436723 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:23.442099 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:23.442074 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:24.128911 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:24.128866 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:24.130324 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:24.130288 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:35.195528 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:35.195491 2562 generic.go:356] "Generic (PLEG): container finished" podID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerID="c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91" exitCode=0 Jul 15 19:43:35.196020 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:35.195569 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" event={"ID":"474e60d7-43b1-46a3-9459-4bfe94d722bb","Type":"ContainerDied","Data":"c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91"} Jul 15 19:43:35.196020 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:35.195995 2562 scope.go:117] "RemoveContainer" containerID="c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91" Jul 15 19:43:35.196975 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:35.196948 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:43:35.197037 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:35.197016 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:43:35.197071 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:35.197054 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:43:36.208143 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:36.208108 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" event={"ID":"474e60d7-43b1-46a3-9459-4bfe94d722bb","Type":"ContainerStarted","Data":"380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0"} Jul 15 19:43:36.436820 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:36.436782 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:36.438084 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:36.438059 2562 patch_prober.go:28] interesting pod/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 15 19:43:36.438216 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:36.438110 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 15 19:43:37.437278 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:37.437240 2562 patch_prober.go:28] interesting pod/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 15 19:43:37.437689 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:37.437295 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 15 19:43:38.436869 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:38.436829 2562 patch_prober.go:28] interesting pod/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 15 19:43:38.437057 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:38.436889 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 15 19:43:39.129719 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:39.129682 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:39.437194 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:39.437155 2562 patch_prober.go:28] interesting pod/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 15 19:43:39.437366 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:39.437226 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 15 19:43:40.440356 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:40.440325 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:43:40.441432 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:43:40.441415 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:44:28.067082 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:28.066983 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6"] Jul 15 19:44:28.067610 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:28.067225 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" containerID="cri-o://380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0" gracePeriod=30 Jul 15 19:44:33.316630 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.316602 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:44:33.432750 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.432704 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-credential-socket\") pod \"474e60d7-43b1-46a3-9459-4bfe94d722bb\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " Jul 15 19:44:33.432960 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.432787 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lds2l\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-kube-api-access-lds2l\") pod \"474e60d7-43b1-46a3-9459-4bfe94d722bb\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " Jul 15 19:44:33.432960 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.432818 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-socket\") pod \"474e60d7-43b1-46a3-9459-4bfe94d722bb\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " Jul 15 19:44:33.432960 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.432869 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/474e60d7-43b1-46a3-9459-4bfe94d722bb-istiod-ca-cert\") pod \"474e60d7-43b1-46a3-9459-4bfe94d722bb\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " Jul 15 19:44:33.432960 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.432916 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-envoy\") pod \"474e60d7-43b1-46a3-9459-4bfe94d722bb\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " Jul 15 19:44:33.432960 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.432960 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-podinfo\") pod \"474e60d7-43b1-46a3-9459-4bfe94d722bb\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " Jul 15 19:44:33.433214 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.432957 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "474e60d7-43b1-46a3-9459-4bfe94d722bb" (UID: "474e60d7-43b1-46a3-9459-4bfe94d722bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:44:33.433214 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.432986 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-token\") pod \"474e60d7-43b1-46a3-9459-4bfe94d722bb\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " Jul 15 19:44:33.433214 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.433034 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-data\") pod \"474e60d7-43b1-46a3-9459-4bfe94d722bb\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " Jul 15 19:44:33.433214 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.433058 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-certs\") pod \"474e60d7-43b1-46a3-9459-4bfe94d722bb\" (UID: \"474e60d7-43b1-46a3-9459-4bfe94d722bb\") " Jul 15 19:44:33.433399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.433224 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "474e60d7-43b1-46a3-9459-4bfe94d722bb" (UID: "474e60d7-43b1-46a3-9459-4bfe94d722bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:44:33.433399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.433393 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-credential-socket\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:44:33.433510 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.433394 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "474e60d7-43b1-46a3-9459-4bfe94d722bb" (UID: "474e60d7-43b1-46a3-9459-4bfe94d722bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:44:33.433510 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.433393 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/474e60d7-43b1-46a3-9459-4bfe94d722bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "474e60d7-43b1-46a3-9459-4bfe94d722bb" (UID: "474e60d7-43b1-46a3-9459-4bfe94d722bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:44:33.433510 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.433413 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-socket\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:44:33.434246 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.434213 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "474e60d7-43b1-46a3-9459-4bfe94d722bb" (UID: "474e60d7-43b1-46a3-9459-4bfe94d722bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:44:33.435224 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.435194 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-kube-api-access-lds2l" (OuterVolumeSpecName: "kube-api-access-lds2l") pod "474e60d7-43b1-46a3-9459-4bfe94d722bb" (UID: "474e60d7-43b1-46a3-9459-4bfe94d722bb"). InnerVolumeSpecName "kube-api-access-lds2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:44:33.435335 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.435322 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "474e60d7-43b1-46a3-9459-4bfe94d722bb" (UID: "474e60d7-43b1-46a3-9459-4bfe94d722bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:44:33.435471 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.435445 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "474e60d7-43b1-46a3-9459-4bfe94d722bb" (UID: "474e60d7-43b1-46a3-9459-4bfe94d722bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:44:33.435542 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.435503 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "474e60d7-43b1-46a3-9459-4bfe94d722bb" (UID: "474e60d7-43b1-46a3-9459-4bfe94d722bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 19:44:33.522117 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.522080 2562 generic.go:356] "Generic (PLEG): container finished" podID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerID="380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0" exitCode=0 Jul 15 19:44:33.522309 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.522128 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" event={"ID":"474e60d7-43b1-46a3-9459-4bfe94d722bb","Type":"ContainerDied","Data":"380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0"} Jul 15 19:44:33.522309 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.522150 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" Jul 15 19:44:33.522309 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.522193 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6" event={"ID":"474e60d7-43b1-46a3-9459-4bfe94d722bb","Type":"ContainerDied","Data":"04018896ace2cae5aed451bc5267c2bed0a5c4ffaf836ddedef6bdada5c1e08d"} Jul 15 19:44:33.522309 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.522216 2562 scope.go:117] "RemoveContainer" containerID="380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0" Jul 15 19:44:33.532447 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.532425 2562 scope.go:117] "RemoveContainer" containerID="c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91" Jul 15 19:44:33.534020 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.533993 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-envoy\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:44:33.534129 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.534028 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-podinfo\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:44:33.534129 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.534041 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-token\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:44:33.534129 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.534054 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-istio-data\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:44:33.534129 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.534067 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/474e60d7-43b1-46a3-9459-4bfe94d722bb-workload-certs\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:44:33.534129 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.534079 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lds2l\" (UniqueName: \"kubernetes.io/projected/474e60d7-43b1-46a3-9459-4bfe94d722bb-kube-api-access-lds2l\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:44:33.534129 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.534093 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/474e60d7-43b1-46a3-9459-4bfe94d722bb-istiod-ca-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:44:33.541242 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.541224 2562 scope.go:117] "RemoveContainer" containerID="380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0" Jul 15 19:44:33.541517 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:44:33.541493 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0\": container with ID starting with 380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0 not found: ID does not exist" containerID="380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0" Jul 15 19:44:33.541736 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.541528 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0"} err="failed to get container status \"380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0\": rpc error: code = NotFound desc = could not find container \"380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0\": container with ID starting with 380bab67f99a3b46d27b5004fef581f3ef82f8a69e4f0c8e97fe6eec8c2763a0 not found: ID does not exist" Jul 15 19:44:33.541736 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.541548 2562 scope.go:117] "RemoveContainer" containerID="c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91" Jul 15 19:44:33.541987 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:44:33.541960 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91\": container with ID starting with c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91 not found: ID does not exist" containerID="c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91" Jul 15 19:44:33.542067 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.541996 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91"} err="failed to get container status \"c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91\": rpc error: code = NotFound desc = could not find container \"c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91\": container with ID starting with c79339ae0dedf4cebc07d8e6024848e3979413321913ae045e9cb523e1e86d91 not found: ID does not exist" Jul 15 19:44:33.545812 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.545789 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6"] Jul 15 19:44:33.551755 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:33.551720 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5827a521-gw-openshift-default-8ccb4b5dc-cz6g6"] Jul 15 19:44:34.744742 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:34.744712 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" path="/var/lib/kubelet/pods/474e60d7-43b1-46a3-9459-4bfe94d722bb/volumes" Jul 15 19:44:40.017651 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.017613 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp"] Jul 15 19:44:40.018261 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.018238 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" Jul 15 19:44:40.018322 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.018264 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" Jul 15 19:44:40.018417 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.018404 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" Jul 15 19:44:40.018478 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.018421 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" Jul 15 19:44:40.022144 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.022118 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.025083 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.025059 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a0aec9a3-openshift-default-dockercfg-vfkb6\"" Jul 15 19:44:40.033905 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.033880 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp"] Jul 15 19:44:40.094469 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.094428 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-data\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.094679 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.094479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-podinfo\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.094679 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.094502 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-socket\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.094679 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.094543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-credential-socket\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.094679 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.094605 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-envoy\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.094679 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.094633 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk4lz\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-kube-api-access-jk4lz\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.094679 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.094650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-token\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.094977 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.094717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istiod-ca-cert\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.094977 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.094815 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-certs\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195326 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195287 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-podinfo\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195553 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195334 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-socket\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195553 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195369 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-credential-socket\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195553 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-envoy\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195553 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195444 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk4lz\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-kube-api-access-jk4lz\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195553 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-token\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195553 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istiod-ca-cert\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195909 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195648 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-certs\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195909 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-data\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.195909 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.195821 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-credential-socket\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.196130 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.196003 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-certs\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.196130 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.196100 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-socket\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.196220 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.196132 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-data\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.196303 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.196283 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istiod-ca-cert\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.197890 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.197872 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-envoy\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.197957 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.197941 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-podinfo\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.204025 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.204001 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-token\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.204227 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.204204 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk4lz\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-kube-api-access-jk4lz\") pod \"e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.333720 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.333636 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:40.472500 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.472477 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp"] Jul 15 19:44:40.474416 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:44:40.474383 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edd2766_7c13_4a5c_b94e_d432cfe1915c.slice/crio-4b466e4ed0427123f6ddc44c2d5a741e4b945e7b28fc2bbb6761a805146be2e8 WatchSource:0}: Error finding container 4b466e4ed0427123f6ddc44c2d5a741e4b945e7b28fc2bbb6761a805146be2e8: Status 404 returned error can't find the container with id 4b466e4ed0427123f6ddc44c2d5a741e4b945e7b28fc2bbb6761a805146be2e8 Jul 15 19:44:40.476614 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.476566 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:44:40.476696 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.476646 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:44:40.476696 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.476673 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:44:40.563774 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.563729 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" event={"ID":"1edd2766-7c13-4a5c-b94e-d432cfe1915c","Type":"ContainerStarted","Data":"efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1"} Jul 15 19:44:40.563931 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.563793 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" event={"ID":"1edd2766-7c13-4a5c-b94e-d432cfe1915c","Type":"ContainerStarted","Data":"4b466e4ed0427123f6ddc44c2d5a741e4b945e7b28fc2bbb6761a805146be2e8"} Jul 15 19:44:40.583679 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:40.583620 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" podStartSLOduration=1.583598137 podStartE2EDuration="1.583598137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:44:40.582353888 +0000 UTC m=+1726.419058664" watchObservedRunningTime="2026-07-15 19:44:40.583598137 +0000 UTC m=+1726.420302914" Jul 15 19:44:41.334430 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:41.334381 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:41.348623 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:41.348566 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:41.569333 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:41.569306 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:41.570364 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:41.570347 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:56.570660 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:56.570553 2562 patch_prober.go:28] interesting pod/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 15 19:44:56.570660 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:56.570644 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 15 19:44:56.658686 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:56.658653 2562 generic.go:356] "Generic (PLEG): container finished" podID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerID="efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1" exitCode=0 Jul 15 19:44:56.658855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:56.658703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" event={"ID":"1edd2766-7c13-4a5c-b94e-d432cfe1915c","Type":"ContainerDied","Data":"efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1"} Jul 15 19:44:56.659127 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:56.659113 2562 scope.go:117] "RemoveContainer" containerID="efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1" Jul 15 19:44:56.660093 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:56.660065 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:44:56.660147 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:56.660133 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:44:56.660185 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:56.660162 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 19:44:57.666179 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:57.666139 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" event={"ID":"1edd2766-7c13-4a5c-b94e-d432cfe1915c","Type":"ContainerStarted","Data":"c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba"} Jul 15 19:44:58.334007 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:58.333966 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:44:58.334254 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:58.334225 2562 patch_prober.go:28] interesting pod/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 15 19:44:58.334336 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:58.334305 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 15 19:44:59.334101 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:59.334072 2562 patch_prober.go:28] interesting pod/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 15 19:44:59.334535 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:44:59.334135 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 15 19:45:00.334941 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:00.334903 2562 patch_prober.go:28] interesting pod/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 15 19:45:00.335311 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:00.334962 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 15 19:45:01.334147 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:01.334105 2562 patch_prober.go:28] interesting pod/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 15 19:45:01.334336 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:01.334179 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 15 19:45:02.338281 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:02.338250 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:45:02.338645 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:02.338513 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:45:02.339198 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:02.339177 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:45:52.944302 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:52.944266 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp"] Jul 15 19:45:52.944746 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:52.944477 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" containerID="cri-o://c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba" gracePeriod=30 Jul 15 19:45:54.757204 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:54.757177 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:45:54.761202 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:54.761180 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:45:58.199115 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.199083 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:45:58.225084 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.224958 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-data\") pod \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " Jul 15 19:45:58.225084 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.225034 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-podinfo\") pod \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " Jul 15 19:45:58.225084 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.225068 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-credential-socket\") pod \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " Jul 15 19:45:58.225495 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.225095 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-socket\") pod \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " Jul 15 19:45:58.225495 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.225152 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-envoy\") pod \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " Jul 15 19:45:58.225495 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.225179 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk4lz\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-kube-api-access-jk4lz\") pod \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " Jul 15 19:45:58.225495 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.225200 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istiod-ca-cert\") pod \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " Jul 15 19:45:58.225495 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.225239 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-certs\") pod \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " Jul 15 19:45:58.225495 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.225276 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-token\") pod \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\" (UID: \"1edd2766-7c13-4a5c-b94e-d432cfe1915c\") " Jul 15 19:45:58.226146 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.226109 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1edd2766-7c13-4a5c-b94e-d432cfe1915c" (UID: "1edd2766-7c13-4a5c-b94e-d432cfe1915c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:45:58.226966 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.226941 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-data" (OuterVolumeSpecName: "istio-data") pod "1edd2766-7c13-4a5c-b94e-d432cfe1915c" (UID: "1edd2766-7c13-4a5c-b94e-d432cfe1915c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:45:58.227486 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.227199 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1edd2766-7c13-4a5c-b94e-d432cfe1915c" (UID: "1edd2766-7c13-4a5c-b94e-d432cfe1915c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:45:58.228033 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.227996 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1edd2766-7c13-4a5c-b94e-d432cfe1915c" (UID: "1edd2766-7c13-4a5c-b94e-d432cfe1915c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:45:58.228324 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.228291 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1edd2766-7c13-4a5c-b94e-d432cfe1915c" (UID: "1edd2766-7c13-4a5c-b94e-d432cfe1915c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:45:58.230832 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.229957 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-token" (OuterVolumeSpecName: "istio-token") pod "1edd2766-7c13-4a5c-b94e-d432cfe1915c" (UID: "1edd2766-7c13-4a5c-b94e-d432cfe1915c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:45:58.230832 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.230792 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1edd2766-7c13-4a5c-b94e-d432cfe1915c" (UID: "1edd2766-7c13-4a5c-b94e-d432cfe1915c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 19:45:58.231490 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.231457 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1edd2766-7c13-4a5c-b94e-d432cfe1915c" (UID: "1edd2766-7c13-4a5c-b94e-d432cfe1915c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:45:58.231725 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.231701 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-kube-api-access-jk4lz" (OuterVolumeSpecName: "kube-api-access-jk4lz") pod "1edd2766-7c13-4a5c-b94e-d432cfe1915c" (UID: "1edd2766-7c13-4a5c-b94e-d432cfe1915c"). InnerVolumeSpecName "kube-api-access-jk4lz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:45:58.326699 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.326668 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-token\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:45:58.326699 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.326697 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-data\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:45:58.326699 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.326706 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-podinfo\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:45:58.326945 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.326714 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-credential-socket\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:45:58.326945 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.326729 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-socket\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:45:58.326945 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.326742 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istio-envoy\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:45:58.326945 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.326754 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk4lz\" (UniqueName: \"kubernetes.io/projected/1edd2766-7c13-4a5c-b94e-d432cfe1915c-kube-api-access-jk4lz\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:45:58.326945 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.326766 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1edd2766-7c13-4a5c-b94e-d432cfe1915c-istiod-ca-cert\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:45:58.326945 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:58.326779 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1edd2766-7c13-4a5c-b94e-d432cfe1915c-workload-certs\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:45:59.001893 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.001804 2562 generic.go:356] "Generic (PLEG): container finished" podID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerID="c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba" exitCode=0 Jul 15 19:45:59.001893 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.001854 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" Jul 15 19:45:59.002065 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.001903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" event={"ID":"1edd2766-7c13-4a5c-b94e-d432cfe1915c","Type":"ContainerDied","Data":"c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba"} Jul 15 19:45:59.002065 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.001960 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp" event={"ID":"1edd2766-7c13-4a5c-b94e-d432cfe1915c","Type":"ContainerDied","Data":"4b466e4ed0427123f6ddc44c2d5a741e4b945e7b28fc2bbb6761a805146be2e8"} Jul 15 19:45:59.002065 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.001991 2562 scope.go:117] "RemoveContainer" containerID="c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba" Jul 15 19:45:59.011374 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.011355 2562 scope.go:117] "RemoveContainer" containerID="efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1" Jul 15 19:45:59.020282 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.020254 2562 scope.go:117] "RemoveContainer" containerID="c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba" Jul 15 19:45:59.020674 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:45:59.020639 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba\": container with ID starting with c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba not found: ID does not exist" containerID="c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba" Jul 15 19:45:59.020770 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.020675 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba"} err="failed to get container status \"c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba\": rpc error: code = NotFound desc = could not find container \"c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba\": container with ID starting with c61349c6088cc3b680db03ef52dca505bcf88c3332e9a580789254c006bac0ba not found: ID does not exist" Jul 15 19:45:59.020770 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.020693 2562 scope.go:117] "RemoveContainer" containerID="efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1" Jul 15 19:45:59.021034 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:45:59.021010 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1\": container with ID starting with efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1 not found: ID does not exist" containerID="efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1" Jul 15 19:45:59.021097 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.021044 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1"} err="failed to get container status \"efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1\": rpc error: code = NotFound desc = could not find container \"efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1\": container with ID starting with efa006b73f852cc7d97db5b3afee3cc057970fcd3583d47c41e8683ad89d53c1 not found: ID does not exist" Jul 15 19:45:59.025449 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.025427 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp"] Jul 15 19:45:59.033535 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:45:59.033508 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a0aec9a3-openshift-default-57d7b48fb9-j9dsp"] Jul 15 19:46:00.254290 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.254264 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4"] Jul 15 19:46:00.254740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.254670 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" Jul 15 19:46:00.254740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.254683 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" Jul 15 19:46:00.254740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.254694 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" Jul 15 19:46:00.254740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.254699 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="474e60d7-43b1-46a3-9459-4bfe94d722bb" containerName="istio-proxy" Jul 15 19:46:00.254740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.254716 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" Jul 15 19:46:00.254740 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.254721 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" Jul 15 19:46:00.254959 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.254778 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" Jul 15 19:46:00.254959 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.254784 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" containerName="istio-proxy" Jul 15 19:46:00.259254 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.259235 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.262022 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.262005 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1ebae1b1-serving-cert\"" Jul 15 19:46:00.272346 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.272318 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4"] Jul 15 19:46:00.345136 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.345094 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxxmx\" (UniqueName: \"kubernetes.io/projected/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-kube-api-access-fxxmx\") pod \"maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.345364 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.345291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-openshift-service-ca\") pod \"maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.345364 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.345348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-maas-api-tls\") pod \"maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.446827 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.446799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-openshift-service-ca\") pod \"maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.446827 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.446839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-maas-api-tls\") pod \"maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.447059 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.446882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxxmx\" (UniqueName: \"kubernetes.io/projected/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-kube-api-access-fxxmx\") pod \"maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.447525 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.447503 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-openshift-service-ca\") pod \"maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.449495 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.449464 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-maas-api-tls\") pod \"maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.457531 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.457506 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxxmx\" (UniqueName: \"kubernetes.io/projected/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-kube-api-access-fxxmx\") pod \"maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.571164 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.571086 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:00.713855 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.713829 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4"] Jul 15 19:46:00.716092 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:46:00.716061 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9161d8_eddb_48e2_9195_dfc0f52af6d5.slice/crio-95991bac94c05e98c326c0cb29a7d6d5226a849f186ef27cecc912ad3e35dd2d WatchSource:0}: Error finding container 95991bac94c05e98c326c0cb29a7d6d5226a849f186ef27cecc912ad3e35dd2d: Status 404 returned error can't find the container with id 95991bac94c05e98c326c0cb29a7d6d5226a849f186ef27cecc912ad3e35dd2d Jul 15 19:46:00.717604 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.717565 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:46:00.743645 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:00.743612 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1edd2766-7c13-4a5c-b94e-d432cfe1915c" path="/var/lib/kubelet/pods/1edd2766-7c13-4a5c-b94e-d432cfe1915c/volumes" Jul 15 19:46:01.015800 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:01.015767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" event={"ID":"ff9161d8-eddb-48e2-9195-dfc0f52af6d5","Type":"ContainerStarted","Data":"95991bac94c05e98c326c0cb29a7d6d5226a849f186ef27cecc912ad3e35dd2d"} Jul 15 19:46:02.023859 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:02.023825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" event={"ID":"ff9161d8-eddb-48e2-9195-dfc0f52af6d5","Type":"ContainerStarted","Data":"ca8017c77af955c4c1dc08d3494fe9a877e3b5f5e84b1a46fa86c3f928b66ae0"} Jul 15 19:46:02.024243 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:02.023893 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:46:02.042079 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:02.042031 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" podStartSLOduration=1.663380585 podStartE2EDuration="2.042014713s" totalImagesPullingTime="378.634128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:46:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:46:02.041024873 +0000 UTC m=+1807.877729648" watchObservedRunningTime="2026-07-15 19:46:02.042014713 +0000 UTC m=+1807.878719545" Jul 15 19:46:08.035459 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:46:08.035424 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:47:05.136568 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.136514 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4"] Jul 15 19:47:05.137098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.136824 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" podUID="ff9161d8-eddb-48e2-9195-dfc0f52af6d5" containerName="maas-api" containerID="cri-o://ca8017c77af955c4c1dc08d3494fe9a877e3b5f5e84b1a46fa86c3f928b66ae0" gracePeriod=30 Jul 15 19:47:05.375148 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.375114 2562 generic.go:356] "Generic (PLEG): container finished" podID="ff9161d8-eddb-48e2-9195-dfc0f52af6d5" containerID="ca8017c77af955c4c1dc08d3494fe9a877e3b5f5e84b1a46fa86c3f928b66ae0" exitCode=0 Jul 15 19:47:05.375225 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.375184 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" event={"ID":"ff9161d8-eddb-48e2-9195-dfc0f52af6d5","Type":"ContainerDied","Data":"ca8017c77af955c4c1dc08d3494fe9a877e3b5f5e84b1a46fa86c3f928b66ae0"} Jul 15 19:47:05.396073 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.396007 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:47:05.525091 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.525057 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-maas-api-tls\") pod \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " Jul 15 19:47:05.525295 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.525129 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-openshift-service-ca\") pod \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " Jul 15 19:47:05.525295 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.525208 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxxmx\" (UniqueName: \"kubernetes.io/projected/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-kube-api-access-fxxmx\") pod \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\" (UID: \"ff9161d8-eddb-48e2-9195-dfc0f52af6d5\") " Jul 15 19:47:05.525556 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.525526 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ff9161d8-eddb-48e2-9195-dfc0f52af6d5" (UID: "ff9161d8-eddb-48e2-9195-dfc0f52af6d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:47:05.527560 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.527534 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ff9161d8-eddb-48e2-9195-dfc0f52af6d5" (UID: "ff9161d8-eddb-48e2-9195-dfc0f52af6d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:47:05.527673 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.527651 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-kube-api-access-fxxmx" (OuterVolumeSpecName: "kube-api-access-fxxmx") pod "ff9161d8-eddb-48e2-9195-dfc0f52af6d5" (UID: "ff9161d8-eddb-48e2-9195-dfc0f52af6d5"). InnerVolumeSpecName "kube-api-access-fxxmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:47:05.626424 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.626378 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxxmx\" (UniqueName: \"kubernetes.io/projected/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-kube-api-access-fxxmx\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:47:05.626424 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.626415 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:47:05.626424 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:05.626424 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff9161d8-eddb-48e2-9195-dfc0f52af6d5-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:47:06.383045 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:06.383015 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" Jul 15 19:47:06.383460 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:06.383009 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4" event={"ID":"ff9161d8-eddb-48e2-9195-dfc0f52af6d5","Type":"ContainerDied","Data":"95991bac94c05e98c326c0cb29a7d6d5226a849f186ef27cecc912ad3e35dd2d"} Jul 15 19:47:06.383460 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:06.383154 2562 scope.go:117] "RemoveContainer" containerID="ca8017c77af955c4c1dc08d3494fe9a877e3b5f5e84b1a46fa86c3f928b66ae0" Jul 15 19:47:06.405772 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:06.405743 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4"] Jul 15 19:47:06.412045 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:06.412017 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ebae1b1-648ddf9f7d-rk6t4"] Jul 15 19:47:06.743649 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:47:06.743546 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9161d8-eddb-48e2-9195-dfc0f52af6d5" path="/var/lib/kubelet/pods/ff9161d8-eddb-48e2-9195-dfc0f52af6d5/volumes" Jul 15 19:49:52.491643 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.491599 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv"] Jul 15 19:49:52.492207 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.492036 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9161d8-eddb-48e2-9195-dfc0f52af6d5" containerName="maas-api" Jul 15 19:49:52.492207 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.492051 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9161d8-eddb-48e2-9195-dfc0f52af6d5" containerName="maas-api" Jul 15 19:49:52.492207 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.492123 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9161d8-eddb-48e2-9195-dfc0f52af6d5" containerName="maas-api" Jul 15 19:49:52.495326 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.495301 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:52.498469 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.498449 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-80adc0ad-serving-cert\"" Jul 15 19:49:52.506011 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.505983 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv"] Jul 15 19:49:52.625263 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.625162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05837108-f3f4-4c9c-9a03-31589c5c32dd-openshift-service-ca\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:52.625263 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.625233 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05837108-f3f4-4c9c-9a03-31589c5c32dd-maas-api-tls\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:52.625487 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.625319 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p5nf\" (UniqueName: \"kubernetes.io/projected/05837108-f3f4-4c9c-9a03-31589c5c32dd-kube-api-access-4p5nf\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:52.726777 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.726730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p5nf\" (UniqueName: \"kubernetes.io/projected/05837108-f3f4-4c9c-9a03-31589c5c32dd-kube-api-access-4p5nf\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:52.726988 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.726796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05837108-f3f4-4c9c-9a03-31589c5c32dd-openshift-service-ca\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:52.726988 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.726844 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05837108-f3f4-4c9c-9a03-31589c5c32dd-maas-api-tls\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:52.726988 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:49:52.726965 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-serving-cert: secret "maas-api-e2e-mt-80adc0ad-serving-cert" not found Jul 15 19:49:52.727166 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:49:52.727042 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05837108-f3f4-4c9c-9a03-31589c5c32dd-maas-api-tls podName:05837108-f3f4-4c9c-9a03-31589c5c32dd nodeName:}" failed. No retries permitted until 2026-07-15 19:49:53.227018413 +0000 UTC m=+2039.063723188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/05837108-f3f4-4c9c-9a03-31589c5c32dd-maas-api-tls") pod "maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" (UID: "05837108-f3f4-4c9c-9a03-31589c5c32dd") : secret "maas-api-e2e-mt-80adc0ad-serving-cert" not found Jul 15 19:49:52.727449 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.727426 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05837108-f3f4-4c9c-9a03-31589c5c32dd-openshift-service-ca\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:52.739714 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:52.739689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p5nf\" (UniqueName: \"kubernetes.io/projected/05837108-f3f4-4c9c-9a03-31589c5c32dd-kube-api-access-4p5nf\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:53.232122 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:53.232085 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05837108-f3f4-4c9c-9a03-31589c5c32dd-maas-api-tls\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:53.234771 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:53.234746 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05837108-f3f4-4c9c-9a03-31589c5c32dd-maas-api-tls\") pod \"maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:53.407131 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:53.407088 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:53.550615 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:53.547946 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv"] Jul 15 19:49:54.310037 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:54.309946 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" event={"ID":"05837108-f3f4-4c9c-9a03-31589c5c32dd","Type":"ContainerStarted","Data":"163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb"} Jul 15 19:49:54.310037 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:54.310018 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:49:54.310234 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:54.310043 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" event={"ID":"05837108-f3f4-4c9c-9a03-31589c5c32dd","Type":"ContainerStarted","Data":"efc86f9332e26b568610545c34df091aee7fd519a2463a8232ba031fd73872f4"} Jul 15 19:49:54.333935 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:49:54.333877 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" podStartSLOduration=1.878470311 podStartE2EDuration="2.333858541s" totalImagesPullingTime="455.38823ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:49:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:49:54.331363244 +0000 UTC m=+2040.168068020" watchObservedRunningTime="2026-07-15 19:49:54.333858541 +0000 UTC m=+2040.170563318" Jul 15 19:50:00.320526 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:50:00.320492 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:50:54.796466 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:50:54.796433 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:50:54.800939 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:50:54.800917 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:51:06.431783 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:06.431748 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv"] Jul 15 19:51:06.712745 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:06.712632 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" podUID="05837108-f3f4-4c9c-9a03-31589c5c32dd" containerName="maas-api" containerID="cri-o://163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb" gracePeriod=30 Jul 15 19:51:06.960459 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:06.960434 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:51:07.004418 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.004309 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p5nf\" (UniqueName: \"kubernetes.io/projected/05837108-f3f4-4c9c-9a03-31589c5c32dd-kube-api-access-4p5nf\") pod \"05837108-f3f4-4c9c-9a03-31589c5c32dd\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " Jul 15 19:51:07.004418 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.004350 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05837108-f3f4-4c9c-9a03-31589c5c32dd-openshift-service-ca\") pod \"05837108-f3f4-4c9c-9a03-31589c5c32dd\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " Jul 15 19:51:07.004418 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.004380 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05837108-f3f4-4c9c-9a03-31589c5c32dd-maas-api-tls\") pod \"05837108-f3f4-4c9c-9a03-31589c5c32dd\" (UID: \"05837108-f3f4-4c9c-9a03-31589c5c32dd\") " Jul 15 19:51:07.004751 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.004723 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05837108-f3f4-4c9c-9a03-31589c5c32dd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05837108-f3f4-4c9c-9a03-31589c5c32dd" (UID: "05837108-f3f4-4c9c-9a03-31589c5c32dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:51:07.006588 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.006546 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05837108-f3f4-4c9c-9a03-31589c5c32dd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05837108-f3f4-4c9c-9a03-31589c5c32dd" (UID: "05837108-f3f4-4c9c-9a03-31589c5c32dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:51:07.006731 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.006713 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05837108-f3f4-4c9c-9a03-31589c5c32dd-kube-api-access-4p5nf" (OuterVolumeSpecName: "kube-api-access-4p5nf") pod "05837108-f3f4-4c9c-9a03-31589c5c32dd" (UID: "05837108-f3f4-4c9c-9a03-31589c5c32dd"). InnerVolumeSpecName "kube-api-access-4p5nf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:51:07.105492 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.105441 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p5nf\" (UniqueName: \"kubernetes.io/projected/05837108-f3f4-4c9c-9a03-31589c5c32dd-kube-api-access-4p5nf\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:51:07.105492 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.105487 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05837108-f3f4-4c9c-9a03-31589c5c32dd-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:51:07.105492 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.105497 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05837108-f3f4-4c9c-9a03-31589c5c32dd-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:51:07.718893 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.718854 2562 generic.go:356] "Generic (PLEG): container finished" podID="05837108-f3f4-4c9c-9a03-31589c5c32dd" containerID="163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb" exitCode=0 Jul 15 19:51:07.718893 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.718897 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" Jul 15 19:51:07.719386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.718937 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" event={"ID":"05837108-f3f4-4c9c-9a03-31589c5c32dd","Type":"ContainerDied","Data":"163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb"} Jul 15 19:51:07.719386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.718994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv" event={"ID":"05837108-f3f4-4c9c-9a03-31589c5c32dd","Type":"ContainerDied","Data":"efc86f9332e26b568610545c34df091aee7fd519a2463a8232ba031fd73872f4"} Jul 15 19:51:07.719386 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.719018 2562 scope.go:117] "RemoveContainer" containerID="163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb" Jul 15 19:51:07.728671 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.728653 2562 scope.go:117] "RemoveContainer" containerID="163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb" Jul 15 19:51:07.728945 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:51:07.728927 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb\": container with ID starting with 163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb not found: ID does not exist" containerID="163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb" Jul 15 19:51:07.729002 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.728954 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb"} err="failed to get container status \"163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb\": rpc error: code = NotFound desc = could not find container \"163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb\": container with ID starting with 163d4deba96c00c379db66f0235176a9ec675f9a10ae95bb6a21f493380408cb not found: ID does not exist" Jul 15 19:51:07.739327 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.739304 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv"] Jul 15 19:51:07.743952 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:07.743932 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-80adc0ad-8b466d49f-qj8kv"] Jul 15 19:51:08.743407 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:51:08.743375 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05837108-f3f4-4c9c-9a03-31589c5c32dd" path="/var/lib/kubelet/pods/05837108-f3f4-4c9c-9a03-31589c5c32dd/volumes" Jul 15 19:53:13.729604 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.729540 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs"] Jul 15 19:53:13.730219 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.730007 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05837108-f3f4-4c9c-9a03-31589c5c32dd" containerName="maas-api" Jul 15 19:53:13.730219 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.730022 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="05837108-f3f4-4c9c-9a03-31589c5c32dd" containerName="maas-api" Jul 15 19:53:13.730219 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.730092 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="05837108-f3f4-4c9c-9a03-31589c5c32dd" containerName="maas-api" Jul 15 19:53:13.733172 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.733152 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:13.735879 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.735851 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-435e1e-serving-cert\"" Jul 15 19:53:13.746170 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.746137 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs"] Jul 15 19:53:13.779935 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.779900 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcdvc\" (UniqueName: \"kubernetes.io/projected/572ae2c3-b5fb-4798-86c8-81a719fb27f3-kube-api-access-qcdvc\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:13.780136 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.779966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572ae2c3-b5fb-4798-86c8-81a719fb27f3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:13.780136 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.780023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572ae2c3-b5fb-4798-86c8-81a719fb27f3-maas-api-tls\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:13.881712 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.881672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572ae2c3-b5fb-4798-86c8-81a719fb27f3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:13.881712 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.881716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572ae2c3-b5fb-4798-86c8-81a719fb27f3-maas-api-tls\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:13.881991 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.881778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcdvc\" (UniqueName: \"kubernetes.io/projected/572ae2c3-b5fb-4798-86c8-81a719fb27f3-kube-api-access-qcdvc\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:13.881991 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:53:13.881914 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-serving-cert: secret "maas-api-e2e-inf-b-435e1e-serving-cert" not found Jul 15 19:53:13.882109 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:53:13.882000 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/572ae2c3-b5fb-4798-86c8-81a719fb27f3-maas-api-tls podName:572ae2c3-b5fb-4798-86c8-81a719fb27f3 nodeName:}" failed. No retries permitted until 2026-07-15 19:53:14.381982188 +0000 UTC m=+2240.218686959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/572ae2c3-b5fb-4798-86c8-81a719fb27f3-maas-api-tls") pod "maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" (UID: "572ae2c3-b5fb-4798-86c8-81a719fb27f3") : secret "maas-api-e2e-inf-b-435e1e-serving-cert" not found Jul 15 19:53:13.882429 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.882401 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572ae2c3-b5fb-4798-86c8-81a719fb27f3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:13.890644 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:13.890562 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcdvc\" (UniqueName: \"kubernetes.io/projected/572ae2c3-b5fb-4798-86c8-81a719fb27f3-kube-api-access-qcdvc\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:14.387091 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:14.387050 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572ae2c3-b5fb-4798-86c8-81a719fb27f3-maas-api-tls\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:14.389828 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:14.389796 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572ae2c3-b5fb-4798-86c8-81a719fb27f3-maas-api-tls\") pod \"maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:14.645054 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:14.644917 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:14.794320 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:14.794293 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs"] Jul 15 19:53:14.795626 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:53:14.795595 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod572ae2c3_b5fb_4798_86c8_81a719fb27f3.slice/crio-85ac82f8289c6c049616d334e5e14abb034e951a7d6b95bb9caf3c2afd61dfbb WatchSource:0}: Error finding container 85ac82f8289c6c049616d334e5e14abb034e951a7d6b95bb9caf3c2afd61dfbb: Status 404 returned error can't find the container with id 85ac82f8289c6c049616d334e5e14abb034e951a7d6b95bb9caf3c2afd61dfbb Jul 15 19:53:14.797098 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:14.797078 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:53:15.442989 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:15.442900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" event={"ID":"572ae2c3-b5fb-4798-86c8-81a719fb27f3","Type":"ContainerStarted","Data":"5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5"} Jul 15 19:53:15.442989 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:15.442949 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" event={"ID":"572ae2c3-b5fb-4798-86c8-81a719fb27f3","Type":"ContainerStarted","Data":"85ac82f8289c6c049616d334e5e14abb034e951a7d6b95bb9caf3c2afd61dfbb"} Jul 15 19:53:15.442989 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:15.442981 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:53:15.470627 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:15.470564 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" podStartSLOduration=2.11096055 podStartE2EDuration="2.470544976s" totalImagesPullingTime="359.584426ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:53:15.467927994 +0000 UTC m=+2241.304632763" watchObservedRunningTime="2026-07-15 19:53:15.470544976 +0000 UTC m=+2241.307249747" Jul 15 19:53:21.456344 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:53:21.456309 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:54:45.272065 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.271981 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs"] Jul 15 19:54:45.272651 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.272211 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" podUID="572ae2c3-b5fb-4798-86c8-81a719fb27f3" containerName="maas-api" containerID="cri-o://5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5" gracePeriod=30 Jul 15 19:54:45.521276 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.521251 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:54:45.642497 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.642465 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcdvc\" (UniqueName: \"kubernetes.io/projected/572ae2c3-b5fb-4798-86c8-81a719fb27f3-kube-api-access-qcdvc\") pod \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " Jul 15 19:54:45.642689 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.642530 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572ae2c3-b5fb-4798-86c8-81a719fb27f3-openshift-service-ca\") pod \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " Jul 15 19:54:45.642689 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.642565 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572ae2c3-b5fb-4798-86c8-81a719fb27f3-maas-api-tls\") pod \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\" (UID: \"572ae2c3-b5fb-4798-86c8-81a719fb27f3\") " Jul 15 19:54:45.642888 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.642864 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572ae2c3-b5fb-4798-86c8-81a719fb27f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "572ae2c3-b5fb-4798-86c8-81a719fb27f3" (UID: "572ae2c3-b5fb-4798-86c8-81a719fb27f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:54:45.644801 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.644778 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/572ae2c3-b5fb-4798-86c8-81a719fb27f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "572ae2c3-b5fb-4798-86c8-81a719fb27f3" (UID: "572ae2c3-b5fb-4798-86c8-81a719fb27f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:54:45.645063 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.645038 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572ae2c3-b5fb-4798-86c8-81a719fb27f3-kube-api-access-qcdvc" (OuterVolumeSpecName: "kube-api-access-qcdvc") pod "572ae2c3-b5fb-4798-86c8-81a719fb27f3" (UID: "572ae2c3-b5fb-4798-86c8-81a719fb27f3"). InnerVolumeSpecName "kube-api-access-qcdvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:54:45.744267 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.744235 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcdvc\" (UniqueName: \"kubernetes.io/projected/572ae2c3-b5fb-4798-86c8-81a719fb27f3-kube-api-access-qcdvc\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:54:45.744267 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.744264 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/572ae2c3-b5fb-4798-86c8-81a719fb27f3-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:54:45.744267 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.744274 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/572ae2c3-b5fb-4798-86c8-81a719fb27f3-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:54:45.962635 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.962522 2562 generic.go:356] "Generic (PLEG): container finished" podID="572ae2c3-b5fb-4798-86c8-81a719fb27f3" containerID="5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5" exitCode=0 Jul 15 19:54:45.962635 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.962564 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" Jul 15 19:54:45.962635 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.962616 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" event={"ID":"572ae2c3-b5fb-4798-86c8-81a719fb27f3","Type":"ContainerDied","Data":"5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5"} Jul 15 19:54:45.962871 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.962673 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs" event={"ID":"572ae2c3-b5fb-4798-86c8-81a719fb27f3","Type":"ContainerDied","Data":"85ac82f8289c6c049616d334e5e14abb034e951a7d6b95bb9caf3c2afd61dfbb"} Jul 15 19:54:45.962871 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.962702 2562 scope.go:117] "RemoveContainer" containerID="5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5" Jul 15 19:54:45.972533 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.972502 2562 scope.go:117] "RemoveContainer" containerID="5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5" Jul 15 19:54:45.972821 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:54:45.972801 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5\": container with ID starting with 5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5 not found: ID does not exist" containerID="5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5" Jul 15 19:54:45.972890 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.972834 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5"} err="failed to get container status \"5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5\": rpc error: code = NotFound desc = could not find container \"5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5\": container with ID starting with 5985e3770cca5cbd4b3d52b94cd436397eeac6d78d1eb0b0fc37974a344231f5 not found: ID does not exist" Jul 15 19:54:45.995006 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:45.994981 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs"] Jul 15 19:54:46.002361 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:46.002339 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-435e1e-699cbd76d6-jrqzs"] Jul 15 19:54:46.743600 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:54:46.743552 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="572ae2c3-b5fb-4798-86c8-81a719fb27f3" path="/var/lib/kubelet/pods/572ae2c3-b5fb-4798-86c8-81a719fb27f3/volumes" Jul 15 19:55:08.158797 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.158750 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt"] Jul 15 19:55:08.161222 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.159220 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="572ae2c3-b5fb-4798-86c8-81a719fb27f3" containerName="maas-api" Jul 15 19:55:08.161222 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.159235 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="572ae2c3-b5fb-4798-86c8-81a719fb27f3" containerName="maas-api" Jul 15 19:55:08.161222 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.159314 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="572ae2c3-b5fb-4798-86c8-81a719fb27f3" containerName="maas-api" Jul 15 19:55:08.162229 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.162207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.164869 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.164845 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a4ccbf-serving-cert\"" Jul 15 19:55:08.177164 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.177141 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt"] Jul 15 19:55:08.245287 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.245252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06eef56b-e843-4024-a903-83b43c4edb97-openshift-service-ca\") pod \"maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.245469 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.245325 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06eef56b-e843-4024-a903-83b43c4edb97-maas-api-tls\") pod \"maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.245469 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.245357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf2v5\" (UniqueName: \"kubernetes.io/projected/06eef56b-e843-4024-a903-83b43c4edb97-kube-api-access-qf2v5\") pod \"maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.346448 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.346413 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06eef56b-e843-4024-a903-83b43c4edb97-maas-api-tls\") pod \"maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.346448 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.346453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf2v5\" (UniqueName: \"kubernetes.io/projected/06eef56b-e843-4024-a903-83b43c4edb97-kube-api-access-qf2v5\") pod \"maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.346723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.346600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06eef56b-e843-4024-a903-83b43c4edb97-openshift-service-ca\") pod \"maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.347255 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.347186 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06eef56b-e843-4024-a903-83b43c4edb97-openshift-service-ca\") pod \"maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.349074 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.349050 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06eef56b-e843-4024-a903-83b43c4edb97-maas-api-tls\") pod \"maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.354539 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.354516 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf2v5\" (UniqueName: \"kubernetes.io/projected/06eef56b-e843-4024-a903-83b43c4edb97-kube-api-access-qf2v5\") pod \"maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.472628 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.472522 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:08.618481 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:08.618454 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt"] Jul 15 19:55:08.619751 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:55:08.619723 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06eef56b_e843_4024_a903_83b43c4edb97.slice/crio-d701ae69bdcba9ff30b119379b709f366602d3fdc85ac95351ccfb98825566e3 WatchSource:0}: Error finding container d701ae69bdcba9ff30b119379b709f366602d3fdc85ac95351ccfb98825566e3: Status 404 returned error can't find the container with id d701ae69bdcba9ff30b119379b709f366602d3fdc85ac95351ccfb98825566e3 Jul 15 19:55:09.103204 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:09.103156 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" event={"ID":"06eef56b-e843-4024-a903-83b43c4edb97","Type":"ContainerStarted","Data":"58a216423242c1d21edcab638780a6697a75eab8e6bb588a6a36fb57f81b9d5b"} Jul 15 19:55:09.103378 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:09.103221 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" event={"ID":"06eef56b-e843-4024-a903-83b43c4edb97","Type":"ContainerStarted","Data":"d701ae69bdcba9ff30b119379b709f366602d3fdc85ac95351ccfb98825566e3"} Jul 15 19:55:09.103378 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:09.103250 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:09.125066 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:09.125019 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" podStartSLOduration=0.75292626 podStartE2EDuration="1.125001337s" totalImagesPullingTime="372.075077ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:09.122177245 +0000 UTC m=+2354.958882022" watchObservedRunningTime="2026-07-15 19:55:09.125001337 +0000 UTC m=+2354.961706145" Jul 15 19:55:15.115934 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:15.115904 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:55:54.835696 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:54.835659 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:55:54.840369 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:55:54.840343 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 19:56:03.194540 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.194499 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt"] Jul 15 19:56:03.195088 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.194755 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" podUID="06eef56b-e843-4024-a903-83b43c4edb97" containerName="maas-api" containerID="cri-o://58a216423242c1d21edcab638780a6697a75eab8e6bb588a6a36fb57f81b9d5b" gracePeriod=30 Jul 15 19:56:03.413036 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.412999 2562 generic.go:356] "Generic (PLEG): container finished" podID="06eef56b-e843-4024-a903-83b43c4edb97" containerID="58a216423242c1d21edcab638780a6697a75eab8e6bb588a6a36fb57f81b9d5b" exitCode=0 Jul 15 19:56:03.413187 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.413074 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" event={"ID":"06eef56b-e843-4024-a903-83b43c4edb97","Type":"ContainerDied","Data":"58a216423242c1d21edcab638780a6697a75eab8e6bb588a6a36fb57f81b9d5b"} Jul 15 19:56:03.448218 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.448153 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:56:03.546839 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.546801 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qf2v5\" (UniqueName: \"kubernetes.io/projected/06eef56b-e843-4024-a903-83b43c4edb97-kube-api-access-qf2v5\") pod \"06eef56b-e843-4024-a903-83b43c4edb97\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " Jul 15 19:56:03.547035 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.546897 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06eef56b-e843-4024-a903-83b43c4edb97-maas-api-tls\") pod \"06eef56b-e843-4024-a903-83b43c4edb97\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " Jul 15 19:56:03.547035 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.546936 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06eef56b-e843-4024-a903-83b43c4edb97-openshift-service-ca\") pod \"06eef56b-e843-4024-a903-83b43c4edb97\" (UID: \"06eef56b-e843-4024-a903-83b43c4edb97\") " Jul 15 19:56:03.547309 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.547281 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06eef56b-e843-4024-a903-83b43c4edb97-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06eef56b-e843-4024-a903-83b43c4edb97" (UID: "06eef56b-e843-4024-a903-83b43c4edb97"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:56:03.549100 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.549073 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06eef56b-e843-4024-a903-83b43c4edb97-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06eef56b-e843-4024-a903-83b43c4edb97" (UID: "06eef56b-e843-4024-a903-83b43c4edb97"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:56:03.549258 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.549240 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06eef56b-e843-4024-a903-83b43c4edb97-kube-api-access-qf2v5" (OuterVolumeSpecName: "kube-api-access-qf2v5") pod "06eef56b-e843-4024-a903-83b43c4edb97" (UID: "06eef56b-e843-4024-a903-83b43c4edb97"). InnerVolumeSpecName "kube-api-access-qf2v5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:56:03.648527 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.648487 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06eef56b-e843-4024-a903-83b43c4edb97-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:56:03.648527 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.648519 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06eef56b-e843-4024-a903-83b43c4edb97-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:56:03.648527 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:03.648534 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf2v5\" (UniqueName: \"kubernetes.io/projected/06eef56b-e843-4024-a903-83b43c4edb97-kube-api-access-qf2v5\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 19:56:04.419723 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:04.419685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" event={"ID":"06eef56b-e843-4024-a903-83b43c4edb97","Type":"ContainerDied","Data":"d701ae69bdcba9ff30b119379b709f366602d3fdc85ac95351ccfb98825566e3"} Jul 15 19:56:04.420165 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:04.419744 2562 scope.go:117] "RemoveContainer" containerID="58a216423242c1d21edcab638780a6697a75eab8e6bb588a6a36fb57f81b9d5b" Jul 15 19:56:04.420165 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:04.419700 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt" Jul 15 19:56:04.443143 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:04.443106 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt"] Jul 15 19:56:04.447555 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:04.447527 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a4ccbf-74b5987c57-qx5mt"] Jul 15 19:56:04.743642 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:04.743551 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06eef56b-e843-4024-a903-83b43c4edb97" path="/var/lib/kubelet/pods/06eef56b-e843-4024-a903-83b43c4edb97/volumes" Jul 15 19:56:18.641269 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.641176 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2"] Jul 15 19:56:18.641706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.641632 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06eef56b-e843-4024-a903-83b43c4edb97" containerName="maas-api" Jul 15 19:56:18.641706 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.641647 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="06eef56b-e843-4024-a903-83b43c4edb97" containerName="maas-api" Jul 15 19:56:18.641797 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.641720 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="06eef56b-e843-4024-a903-83b43c4edb97" containerName="maas-api" Jul 15 19:56:18.646263 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.646241 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:18.648908 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.648889 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-e8718e-serving-cert\"" Jul 15 19:56:18.658613 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.658570 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2"] Jul 15 19:56:18.682277 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.682246 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2259b6-24cc-4ac5-b497-587429eddf4d-maas-api-tls\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:18.682433 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.682353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-599tt\" (UniqueName: \"kubernetes.io/projected/ca2259b6-24cc-4ac5-b497-587429eddf4d-kube-api-access-599tt\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:18.682433 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.682408 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2259b6-24cc-4ac5-b497-587429eddf4d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:18.783596 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.783524 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-599tt\" (UniqueName: \"kubernetes.io/projected/ca2259b6-24cc-4ac5-b497-587429eddf4d-kube-api-access-599tt\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:18.783832 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.783667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2259b6-24cc-4ac5-b497-587429eddf4d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:18.783832 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.783703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2259b6-24cc-4ac5-b497-587429eddf4d-maas-api-tls\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:18.783951 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:56:18.783893 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-serving-cert: secret "maas-api-e2e-shared-a-e8718e-serving-cert" not found Jul 15 19:56:18.784008 ip-10-0-140-236 kubenswrapper[2562]: E0715 19:56:18.783994 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca2259b6-24cc-4ac5-b497-587429eddf4d-maas-api-tls podName:ca2259b6-24cc-4ac5-b497-587429eddf4d nodeName:}" failed. No retries permitted until 2026-07-15 19:56:19.283969429 +0000 UTC m=+2425.120674185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca2259b6-24cc-4ac5-b497-587429eddf4d-maas-api-tls") pod "maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" (UID: "ca2259b6-24cc-4ac5-b497-587429eddf4d") : secret "maas-api-e2e-shared-a-e8718e-serving-cert" not found Jul 15 19:56:18.784441 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.784422 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2259b6-24cc-4ac5-b497-587429eddf4d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:18.795439 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:18.795406 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-599tt\" (UniqueName: \"kubernetes.io/projected/ca2259b6-24cc-4ac5-b497-587429eddf4d-kube-api-access-599tt\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:19.287902 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:19.287849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2259b6-24cc-4ac5-b497-587429eddf4d-maas-api-tls\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:19.290619 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:19.290591 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2259b6-24cc-4ac5-b497-587429eddf4d-maas-api-tls\") pod \"maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:19.557823 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:19.557730 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:19.709061 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:19.709018 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2"] Jul 15 19:56:19.710569 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:56:19.710539 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca2259b6_24cc_4ac5_b497_587429eddf4d.slice/crio-43a4cc92e4ea55ca5987d5aa65ca5bbd4160a1ababdaae8829b4111c648a983a WatchSource:0}: Error finding container 43a4cc92e4ea55ca5987d5aa65ca5bbd4160a1ababdaae8829b4111c648a983a: Status 404 returned error can't find the container with id 43a4cc92e4ea55ca5987d5aa65ca5bbd4160a1ababdaae8829b4111c648a983a Jul 15 19:56:20.513942 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:20.513900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" event={"ID":"ca2259b6-24cc-4ac5-b497-587429eddf4d","Type":"ContainerStarted","Data":"dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a"} Jul 15 19:56:20.514129 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:20.514002 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" event={"ID":"ca2259b6-24cc-4ac5-b497-587429eddf4d","Type":"ContainerStarted","Data":"43a4cc92e4ea55ca5987d5aa65ca5bbd4160a1ababdaae8829b4111c648a983a"} Jul 15 19:56:20.514129 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:20.514037 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:56:20.533429 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:20.533375 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" podStartSLOduration=2.16975904 podStartE2EDuration="2.533355413s" totalImagesPullingTime="363.596373ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:20.533246174 +0000 UTC m=+2426.369950946" watchObservedRunningTime="2026-07-15 19:56:20.533355413 +0000 UTC m=+2426.370060192" Jul 15 19:56:26.524756 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:56:26.524727 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 19:59:13.342553 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.342512 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs"] Jul 15 19:59:13.346040 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.346022 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.350226 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.350204 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"default-dockercfg-5ggtc\"" Jul 15 19:59:13.350361 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.350273 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"openshift-service-ca.crt\"" Jul 15 19:59:13.350361 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.350284 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"sub-test-model-e8718e-kserve-self-signed-certs\"" Jul 15 19:59:13.350361 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.350336 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e8718e\"/\"kube-root-ca.crt\"" Jul 15 19:59:13.354687 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.354668 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs"] Jul 15 19:59:13.383726 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.383669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-dshm\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.383906 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.383736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpzkc\" (UniqueName: \"kubernetes.io/projected/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-kube-api-access-kpzkc\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.383906 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.383770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-model-cache\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.383906 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.383803 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tmp-dir\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.383906 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.383890 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-home\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.384062 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.383931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tls-certs\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.484399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.484363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tmp-dir\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.484399 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.484399 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-home\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.484685 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.484423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tls-certs\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.484685 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.484468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-dshm\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.484685 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.484491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpzkc\" (UniqueName: \"kubernetes.io/projected/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-kube-api-access-kpzkc\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.484685 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.484652 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-model-cache\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.484968 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.484941 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tmp-dir\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.485067 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.484954 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-home\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.485067 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.484998 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-model-cache\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.487006 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.486973 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-dshm\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.487143 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.487126 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tls-certs\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.493640 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.493619 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpzkc\" (UniqueName: \"kubernetes.io/projected/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-kube-api-access-kpzkc\") pod \"sub-test-model-e8718e-kserve-58b4959595-96fhs\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.656646 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.656603 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:13.798168 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:13.798144 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs"] Jul 15 19:59:13.800044 ip-10-0-140-236 kubenswrapper[2562]: W0715 19:59:13.800017 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff99b2c7_0cf9_4ebb_a079_210e5d579bb2.slice/crio-f257945de1a37f8c22372313b6e4507b46fdc80ac815694be6eb20369eec13a0 WatchSource:0}: Error finding container f257945de1a37f8c22372313b6e4507b46fdc80ac815694be6eb20369eec13a0: Status 404 returned error can't find the container with id f257945de1a37f8c22372313b6e4507b46fdc80ac815694be6eb20369eec13a0 Jul 15 19:59:14.491492 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:14.491457 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" event={"ID":"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2","Type":"ContainerStarted","Data":"e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b"} Jul 15 19:59:14.491492 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:14.491501 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" event={"ID":"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2","Type":"ContainerStarted","Data":"f257945de1a37f8c22372313b6e4507b46fdc80ac815694be6eb20369eec13a0"} Jul 15 19:59:14.510063 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:14.510007 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" podStartSLOduration=1.5099891859999999 podStartE2EDuration="1.509989186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:59:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:59:14.509891441 +0000 UTC m=+2600.346596221" watchObservedRunningTime="2026-07-15 19:59:14.509989186 +0000 UTC m=+2600.346693945" Jul 15 19:59:23.656940 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:23.656899 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:23.657338 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:23.656955 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:23.669309 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:23.669281 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 19:59:24.563006 ip-10-0-140-236 kubenswrapper[2562]: I0715 19:59:24.562976 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 20:00:18.169419 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:18.169385 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs"] Jul 15 20:00:33.194269 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.194219 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" podUID="ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" containerName="main" containerID="cri-o://e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b" gracePeriod=45 Jul 15 20:00:33.440975 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.440947 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 20:00:33.543681 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.543571 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpzkc\" (UniqueName: \"kubernetes.io/projected/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-kube-api-access-kpzkc\") pod \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " Jul 15 20:00:33.543681 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.543659 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-home\") pod \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " Jul 15 20:00:33.543907 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.543695 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tls-certs\") pod \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " Jul 15 20:00:33.543907 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.543737 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-dshm\") pod \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " Jul 15 20:00:33.543907 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.543765 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-model-cache\") pod \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " Jul 15 20:00:33.543907 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.543850 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tmp-dir\") pod \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\" (UID: \"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2\") " Jul 15 20:00:33.544126 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.543980 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-home" (OuterVolumeSpecName: "home") pod "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" (UID: "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:00:33.544175 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.544122 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" (UID: "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:00:33.544175 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.544147 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" (UID: "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:00:33.544251 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.544227 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-home\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:00:33.546004 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.545982 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-dshm" (OuterVolumeSpecName: "dshm") pod "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" (UID: "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:00:33.546087 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.546021 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-kube-api-access-kpzkc" (OuterVolumeSpecName: "kube-api-access-kpzkc") pod "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" (UID: "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2"). InnerVolumeSpecName "kube-api-access-kpzkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:00:33.546138 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.546093 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" (UID: "ff99b2c7-0cf9-4ebb-a079-210e5d579bb2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:00:33.645546 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.645509 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tmp-dir\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:00:33.645546 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.645540 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpzkc\" (UniqueName: \"kubernetes.io/projected/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-kube-api-access-kpzkc\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:00:33.645546 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.645549 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-tls-certs\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:00:33.645798 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.645560 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-dshm\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:00:33.645798 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.645568 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2-model-cache\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:00:33.936934 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.936895 2562 generic.go:356] "Generic (PLEG): container finished" podID="ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" containerID="e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b" exitCode=0 Jul 15 20:00:33.936934 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.936937 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" Jul 15 20:00:33.937166 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.936963 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" event={"ID":"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2","Type":"ContainerDied","Data":"e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b"} Jul 15 20:00:33.937166 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.936995 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs" event={"ID":"ff99b2c7-0cf9-4ebb-a079-210e5d579bb2","Type":"ContainerDied","Data":"f257945de1a37f8c22372313b6e4507b46fdc80ac815694be6eb20369eec13a0"} Jul 15 20:00:33.937166 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.937011 2562 scope.go:117] "RemoveContainer" containerID="e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b" Jul 15 20:00:33.946520 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.946497 2562 scope.go:117] "RemoveContainer" containerID="e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b" Jul 15 20:00:33.946841 ip-10-0-140-236 kubenswrapper[2562]: E0715 20:00:33.946822 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b\": container with ID starting with e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b not found: ID does not exist" containerID="e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b" Jul 15 20:00:33.946895 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.946850 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b"} err="failed to get container status \"e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b\": rpc error: code = NotFound desc = could not find container \"e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b\": container with ID starting with e521f2c1925d08dd43ed2a5a27851fcf104868923a100cab66a3b7d9084c8a6b not found: ID does not exist" Jul 15 20:00:33.958907 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.958878 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs"] Jul 15 20:00:33.961119 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:33.961093 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e8718e/sub-test-model-e8718e-kserve-58b4959595-96fhs"] Jul 15 20:00:34.743691 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:34.743656 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" path="/var/lib/kubelet/pods/ff99b2c7-0cf9-4ebb-a079-210e5d579bb2/volumes" Jul 15 20:00:54.870955 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:54.870870 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 20:00:54.876702 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:00:54.876681 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9gs6l_b1d40045-d2ca-4b74-8192-248a35efdf7d/ovn-acl-logging/0.log" Jul 15 20:01:56.770399 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:56.770361 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7571-140356346426800"] Jul 15 20:01:56.770929 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:56.770761 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" containerName="main" Jul 15 20:01:56.770929 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:56.770774 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" containerName="main" Jul 15 20:01:56.770929 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:56.770842 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff99b2c7-0cf9-4ebb-a079-210e5d579bb2" containerName="main" Jul 15 20:01:56.773963 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:56.773946 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346426800" Jul 15 20:01:56.776787 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:56.776761 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tsfrm\"" Jul 15 20:01:56.780399 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:56.780373 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346426800"] Jul 15 20:01:56.924796 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:56.924758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nrzj\" (UniqueName: \"kubernetes.io/projected/b787f6fc-37ce-4593-8a29-8a5624c86007-kube-api-access-5nrzj\") pod \"test-curl-7571-140356346426800\" (UID: \"b787f6fc-37ce-4593-8a29-8a5624c86007\") " pod="opendatahub/test-curl-7571-140356346426800" Jul 15 20:01:57.026035 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:57.025908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nrzj\" (UniqueName: \"kubernetes.io/projected/b787f6fc-37ce-4593-8a29-8a5624c86007-kube-api-access-5nrzj\") pod \"test-curl-7571-140356346426800\" (UID: \"b787f6fc-37ce-4593-8a29-8a5624c86007\") " pod="opendatahub/test-curl-7571-140356346426800" Jul 15 20:01:57.036055 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:57.036027 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nrzj\" (UniqueName: \"kubernetes.io/projected/b787f6fc-37ce-4593-8a29-8a5624c86007-kube-api-access-5nrzj\") pod \"test-curl-7571-140356346426800\" (UID: \"b787f6fc-37ce-4593-8a29-8a5624c86007\") " pod="opendatahub/test-curl-7571-140356346426800" Jul 15 20:01:57.084348 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:57.084314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346426800" Jul 15 20:01:57.230306 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:57.230282 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346426800"] Jul 15 20:01:57.232112 ip-10-0-140-236 kubenswrapper[2562]: W0715 20:01:57.232080 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb787f6fc_37ce_4593_8a29_8a5624c86007.slice/crio-5bb37e4ce25f4d0223c29735c3bead809eb2c79fc6520e852ac66b5af0f568f2 WatchSource:0}: Error finding container 5bb37e4ce25f4d0223c29735c3bead809eb2c79fc6520e852ac66b5af0f568f2: Status 404 returned error can't find the container with id 5bb37e4ce25f4d0223c29735c3bead809eb2c79fc6520e852ac66b5af0f568f2 Jul 15 20:01:57.233552 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:57.233531 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 20:01:57.386419 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:57.386382 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346426800" event={"ID":"b787f6fc-37ce-4593-8a29-8a5624c86007","Type":"ContainerStarted","Data":"5bb37e4ce25f4d0223c29735c3bead809eb2c79fc6520e852ac66b5af0f568f2"} Jul 15 20:01:58.394554 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:58.394515 2562 generic.go:356] "Generic (PLEG): container finished" podID="b787f6fc-37ce-4593-8a29-8a5624c86007" containerID="313ec4927cb7af2f48135814655c6a3adefd49717f743df242acd784d84c27b0" exitCode=0 Jul 15 20:01:58.394984 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:58.394605 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346426800" event={"ID":"b787f6fc-37ce-4593-8a29-8a5624c86007","Type":"ContainerDied","Data":"313ec4927cb7af2f48135814655c6a3adefd49717f743df242acd784d84c27b0"} Jul 15 20:01:59.535163 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.535136 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346426800" Jul 15 20:01:59.572369 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.572327 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7571-140356346426800_b787f6fc-37ce-4593-8a29-8a5624c86007/test-curl-7571-140356346426800/0.log" Jul 15 20:01:59.601106 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.601074 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7571-140356346426800"] Jul 15 20:01:59.605289 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.605262 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7571-140356346426800"] Jul 15 20:01:59.649115 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.649081 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nrzj\" (UniqueName: \"kubernetes.io/projected/b787f6fc-37ce-4593-8a29-8a5624c86007-kube-api-access-5nrzj\") pod \"b787f6fc-37ce-4593-8a29-8a5624c86007\" (UID: \"b787f6fc-37ce-4593-8a29-8a5624c86007\") " Jul 15 20:01:59.651396 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.651325 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b787f6fc-37ce-4593-8a29-8a5624c86007-kube-api-access-5nrzj" (OuterVolumeSpecName: "kube-api-access-5nrzj") pod "b787f6fc-37ce-4593-8a29-8a5624c86007" (UID: "b787f6fc-37ce-4593-8a29-8a5624c86007"). InnerVolumeSpecName "kube-api-access-5nrzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:01:59.750073 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.750033 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nrzj\" (UniqueName: \"kubernetes.io/projected/b787f6fc-37ce-4593-8a29-8a5624c86007-kube-api-access-5nrzj\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:01:59.766838 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.766799 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:01:59.767184 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.767171 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b787f6fc-37ce-4593-8a29-8a5624c86007" containerName="test-curl-7571-140356346426800" Jul 15 20:01:59.767227 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.767186 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b787f6fc-37ce-4593-8a29-8a5624c86007" containerName="test-curl-7571-140356346426800" Jul 15 20:01:59.767261 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.767252 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b787f6fc-37ce-4593-8a29-8a5624c86007" containerName="test-curl-7571-140356346426800" Jul 15 20:01:59.770170 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.770155 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:01:59.776873 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.776848 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:01:59.952124 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:01:59.952037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57cb4\" (UniqueName: \"kubernetes.io/projected/e61ff3ab-6e2a-46dc-8d4a-f15979e82c50-kube-api-access-57cb4\") pod \"test-curl-7571-140356346368912\" (UID: \"e61ff3ab-6e2a-46dc-8d4a-f15979e82c50\") " pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:00.053341 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:00.053299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57cb4\" (UniqueName: \"kubernetes.io/projected/e61ff3ab-6e2a-46dc-8d4a-f15979e82c50-kube-api-access-57cb4\") pod \"test-curl-7571-140356346368912\" (UID: \"e61ff3ab-6e2a-46dc-8d4a-f15979e82c50\") " pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:00.062113 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:00.062087 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57cb4\" (UniqueName: \"kubernetes.io/projected/e61ff3ab-6e2a-46dc-8d4a-f15979e82c50-kube-api-access-57cb4\") pod \"test-curl-7571-140356346368912\" (UID: \"e61ff3ab-6e2a-46dc-8d4a-f15979e82c50\") " pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:00.079983 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:00.079951 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:00.217991 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:00.217956 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:02:00.219163 ip-10-0-140-236 kubenswrapper[2562]: W0715 20:02:00.219137 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode61ff3ab_6e2a_46dc_8d4a_f15979e82c50.slice/crio-3f3197a8587c83cb3d34db6f5f697266c75dd72ab84534aa446e7e8bd1e66492 WatchSource:0}: Error finding container 3f3197a8587c83cb3d34db6f5f697266c75dd72ab84534aa446e7e8bd1e66492: Status 404 returned error can't find the container with id 3f3197a8587c83cb3d34db6f5f697266c75dd72ab84534aa446e7e8bd1e66492 Jul 15 20:02:00.408378 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:00.408346 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bb37e4ce25f4d0223c29735c3bead809eb2c79fc6520e852ac66b5af0f568f2" Jul 15 20:02:00.408378 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:00.408348 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346426800" Jul 15 20:02:00.409879 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:00.409844 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346368912" event={"ID":"e61ff3ab-6e2a-46dc-8d4a-f15979e82c50","Type":"ContainerStarted","Data":"3f3197a8587c83cb3d34db6f5f697266c75dd72ab84534aa446e7e8bd1e66492"} Jul 15 20:02:00.420795 ip-10-0-140-236 kubenswrapper[2562]: E0715 20:02:00.420761 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7571-140356346426800\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="b787f6fc-37ce-4593-8a29-8a5624c86007" pod="opendatahub/test-curl-7571-140356346426800" Jul 15 20:02:00.743706 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:00.743665 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b787f6fc-37ce-4593-8a29-8a5624c86007" path="/var/lib/kubelet/pods/b787f6fc-37ce-4593-8a29-8a5624c86007/volumes" Jul 15 20:02:01.416953 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:01.416913 2562 generic.go:356] "Generic (PLEG): container finished" podID="e61ff3ab-6e2a-46dc-8d4a-f15979e82c50" containerID="2b02d1a18e241311d72d7174b3d10848307f59d7a01908ff485321a78f7c3b2b" exitCode=0 Jul 15 20:02:01.417113 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:01.417003 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346368912" event={"ID":"e61ff3ab-6e2a-46dc-8d4a-f15979e82c50","Type":"ContainerDied","Data":"2b02d1a18e241311d72d7174b3d10848307f59d7a01908ff485321a78f7c3b2b"} Jul 15 20:02:02.564838 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.564811 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:02.573667 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.573643 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57cb4\" (UniqueName: \"kubernetes.io/projected/e61ff3ab-6e2a-46dc-8d4a-f15979e82c50-kube-api-access-57cb4\") pod \"e61ff3ab-6e2a-46dc-8d4a-f15979e82c50\" (UID: \"e61ff3ab-6e2a-46dc-8d4a-f15979e82c50\") " Jul 15 20:02:02.575863 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.575834 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e61ff3ab-6e2a-46dc-8d4a-f15979e82c50-kube-api-access-57cb4" (OuterVolumeSpecName: "kube-api-access-57cb4") pod "e61ff3ab-6e2a-46dc-8d4a-f15979e82c50" (UID: "e61ff3ab-6e2a-46dc-8d4a-f15979e82c50"). InnerVolumeSpecName "kube-api-access-57cb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:02:02.585295 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.585271 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7571-140356346368912_e61ff3ab-6e2a-46dc-8d4a-f15979e82c50/test-curl-7571-140356346368912/0.log" Jul 15 20:02:02.609420 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.609384 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:02:02.612995 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.612964 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:02:02.674453 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.674377 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57cb4\" (UniqueName: \"kubernetes.io/projected/e61ff3ab-6e2a-46dc-8d4a-f15979e82c50-kube-api-access-57cb4\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:02:02.736488 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.736454 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:02:02.736862 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.736848 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e61ff3ab-6e2a-46dc-8d4a-f15979e82c50" containerName="test-curl-7571-140356346368912" Jul 15 20:02:02.736913 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.736863 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e61ff3ab-6e2a-46dc-8d4a-f15979e82c50" containerName="test-curl-7571-140356346368912" Jul 15 20:02:02.736948 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.736938 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e61ff3ab-6e2a-46dc-8d4a-f15979e82c50" containerName="test-curl-7571-140356346368912" Jul 15 20:02:02.740076 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.740054 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:02.747563 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.747531 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e61ff3ab-6e2a-46dc-8d4a-f15979e82c50" path="/var/lib/kubelet/pods/e61ff3ab-6e2a-46dc-8d4a-f15979e82c50/volumes" Jul 15 20:02:02.748138 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.748034 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:02:02.775095 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.775058 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnggr\" (UniqueName: \"kubernetes.io/projected/aa042091-53f5-4152-8502-b8b39cbe562f-kube-api-access-nnggr\") pod \"test-curl-7571-140356346368912\" (UID: \"aa042091-53f5-4152-8502-b8b39cbe562f\") " pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:02.875793 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.875752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnggr\" (UniqueName: \"kubernetes.io/projected/aa042091-53f5-4152-8502-b8b39cbe562f-kube-api-access-nnggr\") pod \"test-curl-7571-140356346368912\" (UID: \"aa042091-53f5-4152-8502-b8b39cbe562f\") " pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:02.886011 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:02.885966 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnggr\" (UniqueName: \"kubernetes.io/projected/aa042091-53f5-4152-8502-b8b39cbe562f-kube-api-access-nnggr\") pod \"test-curl-7571-140356346368912\" (UID: \"aa042091-53f5-4152-8502-b8b39cbe562f\") " pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:03.051776 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:03.051684 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:03.189270 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:03.189240 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:02:03.191356 ip-10-0-140-236 kubenswrapper[2562]: W0715 20:02:03.191314 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa042091_53f5_4152_8502_b8b39cbe562f.slice/crio-a28aa3a035e68512df23434d24747e42e3dbcceb59e09eccea7d68c41e3c25c2 WatchSource:0}: Error finding container a28aa3a035e68512df23434d24747e42e3dbcceb59e09eccea7d68c41e3c25c2: Status 404 returned error can't find the container with id a28aa3a035e68512df23434d24747e42e3dbcceb59e09eccea7d68c41e3c25c2 Jul 15 20:02:03.429473 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:03.429430 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346368912" event={"ID":"aa042091-53f5-4152-8502-b8b39cbe562f","Type":"ContainerStarted","Data":"a28aa3a035e68512df23434d24747e42e3dbcceb59e09eccea7d68c41e3c25c2"} Jul 15 20:02:03.430589 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:03.430555 2562 scope.go:117] "RemoveContainer" containerID="2b02d1a18e241311d72d7174b3d10848307f59d7a01908ff485321a78f7c3b2b" Jul 15 20:02:03.430688 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:03.430600 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:04.440146 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:04.440102 2562 generic.go:356] "Generic (PLEG): container finished" podID="aa042091-53f5-4152-8502-b8b39cbe562f" containerID="c85db76f663cf04eb8e116bbb7402c6594929b67b8a0c959d27957ce9b267688" exitCode=0 Jul 15 20:02:04.440533 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:04.440184 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7571-140356346368912" event={"ID":"aa042091-53f5-4152-8502-b8b39cbe562f","Type":"ContainerDied","Data":"c85db76f663cf04eb8e116bbb7402c6594929b67b8a0c959d27957ce9b267688"} Jul 15 20:02:05.578941 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:05.578919 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:05.598000 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:05.597967 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnggr\" (UniqueName: \"kubernetes.io/projected/aa042091-53f5-4152-8502-b8b39cbe562f-kube-api-access-nnggr\") pod \"aa042091-53f5-4152-8502-b8b39cbe562f\" (UID: \"aa042091-53f5-4152-8502-b8b39cbe562f\") " Jul 15 20:02:05.600292 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:05.600258 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa042091-53f5-4152-8502-b8b39cbe562f-kube-api-access-nnggr" (OuterVolumeSpecName: "kube-api-access-nnggr") pod "aa042091-53f5-4152-8502-b8b39cbe562f" (UID: "aa042091-53f5-4152-8502-b8b39cbe562f"). InnerVolumeSpecName "kube-api-access-nnggr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:02:05.601662 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:05.601557 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7571-140356346368912_aa042091-53f5-4152-8502-b8b39cbe562f/test-curl-7571-140356346368912/0.log" Jul 15 20:02:05.625653 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:05.625620 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:02:05.627997 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:05.627973 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7571-140356346368912"] Jul 15 20:02:05.699703 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:05.699617 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnggr\" (UniqueName: \"kubernetes.io/projected/aa042091-53f5-4152-8502-b8b39cbe562f-kube-api-access-nnggr\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:02:06.452091 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:06.452065 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:06.452267 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:06.452065 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a28aa3a035e68512df23434d24747e42e3dbcceb59e09eccea7d68c41e3c25c2" Jul 15 20:02:06.462238 ip-10-0-140-236 kubenswrapper[2562]: E0715 20:02:06.462207 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7571-140356346368912\" is forbidden: User \"system:node:ip-10-0-140-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-236.ec2.internal' and this object" podUID="aa042091-53f5-4152-8502-b8b39cbe562f" pod="opendatahub/test-curl-7571-140356346368912" Jul 15 20:02:06.744561 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:06.744475 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa042091-53f5-4152-8502-b8b39cbe562f" path="/var/lib/kubelet/pods/aa042091-53f5-4152-8502-b8b39cbe562f/volumes" Jul 15 20:02:22.361924 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:22.361887 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2"] Jul 15 20:02:22.362398 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:22.362170 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" podUID="ca2259b6-24cc-4ac5-b497-587429eddf4d" containerName="maas-api" containerID="cri-o://dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a" gracePeriod=30 Jul 15 20:02:22.824796 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:22.824766 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 20:02:22.961871 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:22.961774 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2259b6-24cc-4ac5-b497-587429eddf4d-maas-api-tls\") pod \"ca2259b6-24cc-4ac5-b497-587429eddf4d\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " Jul 15 20:02:22.961871 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:22.961843 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2259b6-24cc-4ac5-b497-587429eddf4d-openshift-service-ca\") pod \"ca2259b6-24cc-4ac5-b497-587429eddf4d\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " Jul 15 20:02:22.962111 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:22.961880 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-599tt\" (UniqueName: \"kubernetes.io/projected/ca2259b6-24cc-4ac5-b497-587429eddf4d-kube-api-access-599tt\") pod \"ca2259b6-24cc-4ac5-b497-587429eddf4d\" (UID: \"ca2259b6-24cc-4ac5-b497-587429eddf4d\") " Jul 15 20:02:22.962238 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:22.962216 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca2259b6-24cc-4ac5-b497-587429eddf4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca2259b6-24cc-4ac5-b497-587429eddf4d" (UID: "ca2259b6-24cc-4ac5-b497-587429eddf4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:02:22.964061 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:22.964037 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2259b6-24cc-4ac5-b497-587429eddf4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca2259b6-24cc-4ac5-b497-587429eddf4d" (UID: "ca2259b6-24cc-4ac5-b497-587429eddf4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 20:02:22.964164 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:22.964144 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca2259b6-24cc-4ac5-b497-587429eddf4d-kube-api-access-599tt" (OuterVolumeSpecName: "kube-api-access-599tt") pod "ca2259b6-24cc-4ac5-b497-587429eddf4d" (UID: "ca2259b6-24cc-4ac5-b497-587429eddf4d"). InnerVolumeSpecName "kube-api-access-599tt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:02:23.063488 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.063452 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca2259b6-24cc-4ac5-b497-587429eddf4d-maas-api-tls\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:02:23.063488 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.063482 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca2259b6-24cc-4ac5-b497-587429eddf4d-openshift-service-ca\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:02:23.063488 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.063493 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-599tt\" (UniqueName: \"kubernetes.io/projected/ca2259b6-24cc-4ac5-b497-587429eddf4d-kube-api-access-599tt\") on node \"ip-10-0-140-236.ec2.internal\" DevicePath \"\"" Jul 15 20:02:23.552871 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.552829 2562 generic.go:356] "Generic (PLEG): container finished" podID="ca2259b6-24cc-4ac5-b497-587429eddf4d" containerID="dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a" exitCode=0 Jul 15 20:02:23.553327 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.552885 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" Jul 15 20:02:23.553327 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.552919 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" event={"ID":"ca2259b6-24cc-4ac5-b497-587429eddf4d","Type":"ContainerDied","Data":"dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a"} Jul 15 20:02:23.553327 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.552975 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2" event={"ID":"ca2259b6-24cc-4ac5-b497-587429eddf4d","Type":"ContainerDied","Data":"43a4cc92e4ea55ca5987d5aa65ca5bbd4160a1ababdaae8829b4111c648a983a"} Jul 15 20:02:23.553327 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.552997 2562 scope.go:117] "RemoveContainer" containerID="dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a" Jul 15 20:02:23.565649 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.564700 2562 scope.go:117] "RemoveContainer" containerID="dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a" Jul 15 20:02:23.565649 ip-10-0-140-236 kubenswrapper[2562]: E0715 20:02:23.565613 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a\": container with ID starting with dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a not found: ID does not exist" containerID="dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a" Jul 15 20:02:23.565649 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.565641 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a"} err="failed to get container status \"dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a\": rpc error: code = NotFound desc = could not find container \"dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a\": container with ID starting with dd28489b721885bfdf8138db81bbe58efd0745f6ec85a8a2c77a852ab359238a not found: ID does not exist" Jul 15 20:02:23.576946 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.576922 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2"] Jul 15 20:02:23.582339 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:23.582312 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e8718e-6ccc85c556-6swf2"] Jul 15 20:02:24.743396 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:24.743361 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca2259b6-24cc-4ac5-b497-587429eddf4d" path="/var/lib/kubelet/pods/ca2259b6-24cc-4ac5-b497-587429eddf4d/volumes" Jul 15 20:02:56.460557 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:02:56.460518 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-548f6cb7d-8f7s6_ca4c8d19-3217-4017-99c7-8bdaf5d4ca38/authorino/0.log" Jul 15 20:03:02.416066 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:02.416030 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cb8df54d8-dzdnd_d09d6a2b-890d-42a4-a0bf-47b61fb821cf/manager/0.log" Jul 15 20:03:02.920519 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:02.920492 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f46f5478c-nfgbm_1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd/copy-manifests/0.log" Jul 15 20:03:02.938660 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:02.938628 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f46f5478c-nfgbm_1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd/manager/0.log" Jul 15 20:03:03.444866 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:03.444818 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6d7b7f858-qsgnf_212d8556-45f8-4904-9343-47c9b5c99bb6/maas-api/0.log" Jul 15 20:03:05.300937 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:05.300903 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wnqc8_4ac389d6-1a82-4453-b0b7-8909bab06a13/postgres/0.log" Jul 15 20:03:06.109670 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.109637 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2_600f7296-13ac-4f3e-a8a8-4a8a6baf11a1/extract/0.log" Jul 15 20:03:06.115324 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.115299 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2_600f7296-13ac-4f3e-a8a8-4a8a6baf11a1/util/0.log" Jul 15 20:03:06.122239 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.122202 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2_600f7296-13ac-4f3e-a8a8-4a8a6baf11a1/pull/0.log" Jul 15 20:03:06.240975 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.240940 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2_d7ef42c9-4e6f-4d38-91b6-6138510417ec/util/0.log" Jul 15 20:03:06.247140 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.247113 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2_d7ef42c9-4e6f-4d38-91b6-6138510417ec/pull/0.log" Jul 15 20:03:06.252864 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.252837 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2_d7ef42c9-4e6f-4d38-91b6-6138510417ec/extract/0.log" Jul 15 20:03:06.378141 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.378062 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb_e96879d9-963c-4145-9ca3-feed596fbd45/pull/0.log" Jul 15 20:03:06.384037 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.384012 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb_e96879d9-963c-4145-9ca3-feed596fbd45/extract/0.log" Jul 15 20:03:06.389833 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.389806 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb_e96879d9-963c-4145-9ca3-feed596fbd45/util/0.log" Jul 15 20:03:06.509862 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.509835 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k_e1c23338-2e3f-4c9f-a770-d669cefbe419/util/0.log" Jul 15 20:03:06.516384 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.516352 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k_e1c23338-2e3f-4c9f-a770-d669cefbe419/pull/0.log" Jul 15 20:03:06.522274 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.522252 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k_e1c23338-2e3f-4c9f-a770-d669cefbe419/extract/0.log" Jul 15 20:03:06.651950 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.651917 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-548f6cb7d-8f7s6_ca4c8d19-3217-4017-99c7-8bdaf5d4ca38/authorino/0.log" Jul 15 20:03:06.909779 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:06.909695 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ps79r_7c822e55-0ece-4872-ab2d-185dde5b050d/manager/0.log" Jul 15 20:03:07.029405 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:07.029376 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zx6jx_d43e74ae-4bec-44ec-a141-889949d3016b/kuadrant-console-plugin/0.log" Jul 15 20:03:07.156426 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:07.156382 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-c4p4r_341502b5-1a44-4fd3-9d39-888cc00721ab/registry-server/0.log" Jul 15 20:03:07.533838 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:07.533802 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wbrgc_0c14da53-96cd-494a-a713-cfac1aab77b5/manager/0.log" Jul 15 20:03:07.908925 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:07.908894 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gjpgj_ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030/istio-proxy/0.log" Jul 15 20:03:08.388077 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:08.388024 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ts9ph_9de8a364-dd9e-47bb-9520-3a51688ff72f/istio-proxy/1.log" Jul 15 20:03:08.514185 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:08.514139 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-lcs6p_7d9003bf-3735-46f8-8470-cb8916dc2316/payload-pre-processing/0.log" Jul 15 20:03:08.644540 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:08.644452 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-2r49c_072c3325-7f97-4c48-9b9d-24470a673bdb/payload-processing/0.log" Jul 15 20:03:09.254727 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:09.254690 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-xsxkq_a17cfcb5-4e21-4d2b-8e9e-757a4fb9d312/main/0.log" Jul 15 20:03:09.486163 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:09.486108 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrwpx8_ff95dfdc-6555-4409-8f2c-ffe7f73938c2/main/0.log" Jul 15 20:03:09.720217 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:09.720190 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-vzz42_219db70a-33ce-4fbe-885b-7491928fdcc1/main/0.log" Jul 15 20:03:18.311240 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:18.311206 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j8jpj_648cd87d-755f-4648-97d3-0857e63a32bb/global-pull-secret-syncer/0.log" Jul 15 20:03:18.492451 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:18.492416 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vxc2j_a87c2704-0fe6-4050-87d0-cc80a8da963b/konnectivity-agent/0.log" Jul 15 20:03:18.542991 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:18.542957 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-236.ec2.internal_df1b6af21dffcdeba9447749a6e14c99/haproxy/0.log" Jul 15 20:03:22.685393 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.685364 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2_600f7296-13ac-4f3e-a8a8-4a8a6baf11a1/extract/0.log" Jul 15 20:03:22.711591 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.711547 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2_600f7296-13ac-4f3e-a8a8-4a8a6baf11a1/util/0.log" Jul 15 20:03:22.737424 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.737393 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lrnf2_600f7296-13ac-4f3e-a8a8-4a8a6baf11a1/pull/0.log" Jul 15 20:03:22.767530 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.767499 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2_d7ef42c9-4e6f-4d38-91b6-6138510417ec/extract/0.log" Jul 15 20:03:22.788550 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.788521 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2_d7ef42c9-4e6f-4d38-91b6-6138510417ec/util/0.log" Jul 15 20:03:22.810536 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.810501 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jgwz2_d7ef42c9-4e6f-4d38-91b6-6138510417ec/pull/0.log" Jul 15 20:03:22.838612 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.838569 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb_e96879d9-963c-4145-9ca3-feed596fbd45/extract/0.log" Jul 15 20:03:22.862474 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.862442 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb_e96879d9-963c-4145-9ca3-feed596fbd45/util/0.log" Jul 15 20:03:22.886852 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.886826 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v5twb_e96879d9-963c-4145-9ca3-feed596fbd45/pull/0.log" Jul 15 20:03:22.910390 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.910362 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k_e1c23338-2e3f-4c9f-a770-d669cefbe419/extract/0.log" Jul 15 20:03:22.940732 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.940647 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k_e1c23338-2e3f-4c9f-a770-d669cefbe419/util/0.log" Jul 15 20:03:22.968419 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:22.968385 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1t695k_e1c23338-2e3f-4c9f-a770-d669cefbe419/pull/0.log" Jul 15 20:03:23.591014 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:23.590978 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-548f6cb7d-8f7s6_ca4c8d19-3217-4017-99c7-8bdaf5d4ca38/authorino/0.log" Jul 15 20:03:23.658890 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:23.658858 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ps79r_7c822e55-0ece-4872-ab2d-185dde5b050d/manager/0.log" Jul 15 20:03:23.686740 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:23.686716 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zx6jx_d43e74ae-4bec-44ec-a141-889949d3016b/kuadrant-console-plugin/0.log" Jul 15 20:03:23.739983 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:23.739943 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-c4p4r_341502b5-1a44-4fd3-9d39-888cc00721ab/registry-server/0.log" Jul 15 20:03:24.101717 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:24.101686 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wbrgc_0c14da53-96cd-494a-a713-cfac1aab77b5/manager/0.log" Jul 15 20:03:25.705194 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:25.705159 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c32b9973-d4e6-408f-83cf-699e25887287/alertmanager/0.log" Jul 15 20:03:25.739295 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:25.739256 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c32b9973-d4e6-408f-83cf-699e25887287/config-reloader/0.log" Jul 15 20:03:25.764567 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:25.764527 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c32b9973-d4e6-408f-83cf-699e25887287/kube-rbac-proxy-web/0.log" Jul 15 20:03:25.785678 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:25.785630 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c32b9973-d4e6-408f-83cf-699e25887287/kube-rbac-proxy/0.log" Jul 15 20:03:25.806691 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:25.806652 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c32b9973-d4e6-408f-83cf-699e25887287/kube-rbac-proxy-metric/0.log" Jul 15 20:03:25.828679 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:25.828654 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c32b9973-d4e6-408f-83cf-699e25887287/prom-label-proxy/0.log" Jul 15 20:03:25.851072 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:25.851040 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c32b9973-d4e6-408f-83cf-699e25887287/init-config-reloader/0.log" Jul 15 20:03:26.087676 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:26.087543 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jjbqn_2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0/node-exporter/0.log" Jul 15 20:03:26.107418 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:26.107394 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jjbqn_2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0/kube-rbac-proxy/0.log" Jul 15 20:03:26.129282 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:26.129255 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jjbqn_2eb8ebd9-ff60-4e9e-aaa1-fe19738e61e0/init-textfile/0.log" Jul 15 20:03:26.691652 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:26.691625 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f4f5f7f69-gb5x2_6f5b4050-a275-46e3-a713-e6cf11b5af5b/telemeter-client/0.log" Jul 15 20:03:26.711829 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:26.711762 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f4f5f7f69-gb5x2_6f5b4050-a275-46e3-a713-e6cf11b5af5b/reload/0.log" Jul 15 20:03:26.731320 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:26.731289 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f4f5f7f69-gb5x2_6f5b4050-a275-46e3-a713-e6cf11b5af5b/kube-rbac-proxy/0.log" Jul 15 20:03:28.101197 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.101166 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w"] Jul 15 20:03:28.101601 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.101521 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2259b6-24cc-4ac5-b497-587429eddf4d" containerName="maas-api" Jul 15 20:03:28.101601 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.101532 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2259b6-24cc-4ac5-b497-587429eddf4d" containerName="maas-api" Jul 15 20:03:28.101601 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.101552 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa042091-53f5-4152-8502-b8b39cbe562f" containerName="test-curl-7571-140356346368912" Jul 15 20:03:28.101601 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.101559 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa042091-53f5-4152-8502-b8b39cbe562f" containerName="test-curl-7571-140356346368912" Jul 15 20:03:28.101764 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.101637 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca2259b6-24cc-4ac5-b497-587429eddf4d" containerName="maas-api" Jul 15 20:03:28.101764 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.101645 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa042091-53f5-4152-8502-b8b39cbe562f" containerName="test-curl-7571-140356346368912" Jul 15 20:03:28.104587 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.104554 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.107406 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.107385 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pp6mc\"/\"openshift-service-ca.crt\"" Jul 15 20:03:28.107547 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.107386 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pp6mc\"/\"default-dockercfg-z9m9k\"" Jul 15 20:03:28.108567 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.108551 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pp6mc\"/\"kube-root-ca.crt\"" Jul 15 20:03:28.115785 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.115757 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w"] Jul 15 20:03:28.250291 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.250254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-lib-modules\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.250291 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.250297 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-sys\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.250519 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.250367 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-proc\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.250519 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.250411 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-podres\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.250519 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.250484 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnmqb\" (UniqueName: \"kubernetes.io/projected/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-kube-api-access-pnmqb\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.351137 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.351098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-lib-modules\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.351137 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.351144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-sys\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.351402 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.351194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-proc\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.351402 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.351229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-podres\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.351402 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.351300 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-lib-modules\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.351402 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.351303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-sys\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.351402 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.351308 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-proc\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.351402 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.351301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnmqb\" (UniqueName: \"kubernetes.io/projected/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-kube-api-access-pnmqb\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.351402 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.351388 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-podres\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.361056 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.361036 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnmqb\" (UniqueName: \"kubernetes.io/projected/bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5-kube-api-access-pnmqb\") pod \"perf-node-gather-daemonset-gvn4w\" (UID: \"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5\") " pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.415907 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.415862 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.554379 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.554348 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w"] Jul 15 20:03:28.555674 ip-10-0-140-236 kubenswrapper[2562]: W0715 20:03:28.555647 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbb8d33e8_2a58_4ae4_a84e_1a93bd3b82a5.slice/crio-dc7ef7e89b47e4538830fd4d15a3709fbc5cabe0835637833d6fc52e33049aa9 WatchSource:0}: Error finding container dc7ef7e89b47e4538830fd4d15a3709fbc5cabe0835637833d6fc52e33049aa9: Status 404 returned error can't find the container with id dc7ef7e89b47e4538830fd4d15a3709fbc5cabe0835637833d6fc52e33049aa9 Jul 15 20:03:28.916300 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.916269 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" event={"ID":"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5","Type":"ContainerStarted","Data":"dc825c8e7768db3c8451e5dd64b98d4929fde73b773992da403a52f0841cea6f"} Jul 15 20:03:28.916553 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.916324 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:28.916553 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.916342 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" event={"ID":"bb8d33e8-2a58-4ae4-a84e-1a93bd3b82a5","Type":"ContainerStarted","Data":"dc7ef7e89b47e4538830fd4d15a3709fbc5cabe0835637833d6fc52e33049aa9"} Jul 15 20:03:28.936278 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:28.936219 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" podStartSLOduration=0.936203157 podStartE2EDuration="936.203157ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:03:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:28.933877552 +0000 UTC m=+2854.770582333" watchObservedRunningTime="2026-07-15 20:03:28.936203157 +0000 UTC m=+2854.772907935" Jul 15 20:03:30.937149 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:30.937116 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dpnpp_8ca0c146-c33d-45e7-9f93-4b004774d54a/dns/0.log" Jul 15 20:03:30.957690 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:30.957665 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dpnpp_8ca0c146-c33d-45e7-9f93-4b004774d54a/kube-rbac-proxy/0.log" Jul 15 20:03:31.031267 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:31.031238 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bcrrg_0a9504f5-8c66-4ec0-a5c3-3406952b5302/dns-node-resolver/0.log" Jul 15 20:03:31.700365 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:31.700332 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-z6fkb_aa06f283-9e57-4543-a309-5605c58b0893/node-ca/0.log" Jul 15 20:03:32.626909 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:32.626868 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gjpgj_ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030/istio-proxy/0.log" Jul 15 20:03:32.835830 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:32.835797 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ts9ph_9de8a364-dd9e-47bb-9520-3a51688ff72f/istio-proxy/0.log" Jul 15 20:03:32.897039 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:32.897003 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ts9ph_9de8a364-dd9e-47bb-9520-3a51688ff72f/istio-proxy/1.log" Jul 15 20:03:32.937340 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:32.937303 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-lcs6p_7d9003bf-3735-46f8-8470-cb8916dc2316/payload-pre-processing/0.log" Jul 15 20:03:32.976939 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:32.976901 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-2r49c_072c3325-7f97-4c48-9b9d-24470a673bdb/payload-processing/0.log" Jul 15 20:03:33.675058 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:33.675028 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m5dcq_86e9f1ce-b8c5-4eed-a402-757a5d74501f/serve-healthcheck-canary/0.log" Jul 15 20:03:34.348493 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:34.348457 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzdk7_56161d04-16b3-4de2-8411-38c3c90b2de4/kube-rbac-proxy/0.log" Jul 15 20:03:34.370458 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:34.370427 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzdk7_56161d04-16b3-4de2-8411-38c3c90b2de4/exporter/0.log" Jul 15 20:03:34.391554 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:34.391524 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lzdk7_56161d04-16b3-4de2-8411-38c3c90b2de4/extractor/0.log" Jul 15 20:03:34.933215 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:34.933183 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pp6mc/perf-node-gather-daemonset-gvn4w" Jul 15 20:03:37.248815 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:37.248785 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cb8df54d8-dzdnd_d09d6a2b-890d-42a4-a0bf-47b61fb821cf/manager/0.log" Jul 15 20:03:37.586355 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:37.586316 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f46f5478c-nfgbm_1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd/manager/0.log" Jul 15 20:03:37.603735 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:37.603701 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f46f5478c-nfgbm_1cb5b76a-c9df-4ed0-ae2b-7b32516ed9bd/copy-manifests/0.log" Jul 15 20:03:38.641032 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:38.640997 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gjpgj_ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030/istio-proxy/0.log" Jul 15 20:03:38.805090 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:38.805056 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ts9ph_9de8a364-dd9e-47bb-9520-3a51688ff72f/istio-proxy/0.log" Jul 15 20:03:38.863158 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:38.863127 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ts9ph_9de8a364-dd9e-47bb-9520-3a51688ff72f/istio-proxy/1.log" Jul 15 20:03:38.900252 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:38.900163 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-lcs6p_7d9003bf-3735-46f8-8470-cb8916dc2316/payload-pre-processing/0.log" Jul 15 20:03:38.929502 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:38.929464 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-2r49c_072c3325-7f97-4c48-9b9d-24470a673bdb/payload-processing/0.log" Jul 15 20:03:38.934092 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:38.934068 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c74497d4c-x7mfn_ffcaa03a-3d64-4bef-be87-2c155f88e9c6/manager/0.log" Jul 15 20:03:38.965147 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:38.965110 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tfknh_c3d40aa3-be12-4bfa-b115-9211dbd18db6/openshift-lws-operator/0.log" Jul 15 20:03:40.223939 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:40.223899 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gjpgj_ba6bcc4b-f4dd-4141-827a-bc9ea8e7b030/istio-proxy/0.log" Jul 15 20:03:40.392982 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:40.392940 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ts9ph_9de8a364-dd9e-47bb-9520-3a51688ff72f/istio-proxy/0.log" Jul 15 20:03:40.451173 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:40.451137 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ts9ph_9de8a364-dd9e-47bb-9520-3a51688ff72f/istio-proxy/1.log" Jul 15 20:03:40.480338 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:40.480240 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-lcs6p_7d9003bf-3735-46f8-8470-cb8916dc2316/payload-pre-processing/0.log" Jul 15 20:03:40.512729 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:40.512694 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-2r49c_072c3325-7f97-4c48-9b9d-24470a673bdb/payload-processing/0.log" Jul 15 20:03:45.452998 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:45.452973 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cdt4d_320343bc-1b3b-4ca7-9ae4-1e073451be95/kube-multus-additional-cni-plugins/0.log" Jul 15 20:03:45.473145 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:45.473117 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cdt4d_320343bc-1b3b-4ca7-9ae4-1e073451be95/egress-router-binary-copy/0.log" Jul 15 20:03:45.493292 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:45.493259 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cdt4d_320343bc-1b3b-4ca7-9ae4-1e073451be95/cni-plugins/0.log" Jul 15 20:03:45.513509 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:45.513482 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cdt4d_320343bc-1b3b-4ca7-9ae4-1e073451be95/bond-cni-plugin/0.log" Jul 15 20:03:45.534159 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:45.534134 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cdt4d_320343bc-1b3b-4ca7-9ae4-1e073451be95/routeoverride-cni/0.log" Jul 15 20:03:45.572821 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:45.572797 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cdt4d_320343bc-1b3b-4ca7-9ae4-1e073451be95/whereabouts-cni-bincopy/0.log" Jul 15 20:03:45.594398 ip-10-0-140-236 kubenswrapper[2562]: I0715 20:03:45.594370 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cdt4d_320343bc-1b3b-4ca7-9ae4-1e073451be95/whereabouts-cni/0.log"