Jul 09 22:06:45.905687 ip-10-0-135-244 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 22:06:45.905697 ip-10-0-135-244 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 22:06:45.905704 ip-10-0-135-244 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 22:06:45.905918 ip-10-0-135-244 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 22:06:55.930503 ip-10-0-135-244 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 22:06:55.930523 ip-10-0-135-244 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 48f1d1fb064b4a87a7510e8db71151c7 -- Jul 09 22:08:51.785011 ip-10-0-135-244 systemd[1]: Starting Kubernetes Kubelet... Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.166688 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170833 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170845 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170848 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170852 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170854 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170857 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170860 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:08:52.253495 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170863 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:08:52.236857 ip-10-0-135-244 systemd[1]: Started Kubernetes Kubelet. Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170865 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170870 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170874 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170878 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170882 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170885 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170890 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170893 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170896 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170899 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170901 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170904 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170906 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170909 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170912 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170915 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170918 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170921 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:08:52.254633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170923 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170926 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170929 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170931 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170934 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170937 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170939 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170943 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170947 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170950 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170953 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170956 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170959 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170962 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170964 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170967 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170970 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170973 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170976 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:08:52.255703 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170978 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170981 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170984 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170987 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170990 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170992 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170995 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.170997 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171000 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171003 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171005 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171008 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171012 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171014 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171017 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171019 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171023 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171026 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171028 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171031 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:08:52.257034 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171034 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171036 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171039 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171041 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171044 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171046 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171049 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171051 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171054 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171057 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171060 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171062 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171065 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171068 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171071 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171075 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171077 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171080 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171082 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171085 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:08:52.258271 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171087 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171090 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171092 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171094 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171097 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171100 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171102 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171105 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171107 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171110 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171114 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171117 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171119 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.171122 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174253 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174265 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174269 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174272 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174276 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174280 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:08:52.258981 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174283 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174286 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174289 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174293 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174298 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174301 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174304 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174307 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174310 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174313 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174315 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174318 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174321 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174324 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174327 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174329 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174332 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174335 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174338 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174341 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:08:52.259979 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174343 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174346 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174349 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174351 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174355 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174357 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174360 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174362 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174365 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174368 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174370 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174373 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174375 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174378 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174380 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174383 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174386 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174389 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174392 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174394 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:08:52.260849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174397 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174400 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174402 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174405 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174408 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174411 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174413 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174416 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174418 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174421 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174423 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174426 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174428 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174431 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174434 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174436 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174439 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174441 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174444 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174446 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:08:52.261807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174449 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174452 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174457 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174460 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174463 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174467 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174469 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174472 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174475 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174478 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174481 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174484 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174488 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174491 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174494 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174497 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174499 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174502 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174504 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:08:52.263236 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174507 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174510 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174512 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174515 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174517 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174520 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174522 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174526 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174528 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174531 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174534 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174537 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174540 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174542 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174950 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174955 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174958 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174961 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174964 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174966 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:08:52.264214 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174969 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174972 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174974 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174979 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174982 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174985 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174987 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174990 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174992 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174995 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.174998 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175001 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175004 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175007 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175010 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175012 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175015 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175018 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175020 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175024 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:08:52.429316 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175027 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175030 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175033 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175036 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175038 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175041 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175044 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175047 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175049 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175052 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175054 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175057 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175059 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175062 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175064 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175068 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175071 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175073 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175077 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:08:52.450712 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175081 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175084 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175087 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175090 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175094 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175097 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175100 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175103 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175105 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175108 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175110 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175113 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175115 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175118 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175120 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175123 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175125 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175127 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175130 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175132 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:08:52.451406 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175135 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175137 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175140 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175143 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175145 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175148 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175151 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175154 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175157 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175160 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175163 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175165 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175168 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175170 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175172 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175175 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175179 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175182 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175184 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175187 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:08:52.452123 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175189 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175191 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175194 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175197 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175199 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175202 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175204 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175208 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175211 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175214 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175217 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175220 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175223 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175226 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175294 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175300 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175305 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175309 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175313 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175316 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175321 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 22:08:52.452756 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175325 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175328 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175332 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175335 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175338 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175341 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175344 2576 flags.go:64] FLAG: --cgroup-root="" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175349 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175352 2576 flags.go:64] FLAG: --client-ca-file="" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175355 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175358 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175361 2576 flags.go:64] FLAG: --cluster-domain="" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175364 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175367 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175370 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175373 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175377 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175380 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175383 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175386 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175389 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175392 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175395 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175398 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 22:08:52.453504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175400 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175404 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175407 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175410 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175413 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175415 2576 flags.go:64] FLAG: --enable-server="true" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175418 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175423 2576 flags.go:64] FLAG: --event-burst="100" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175426 2576 flags.go:64] FLAG: --event-qps="50" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175429 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175432 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175434 2576 flags.go:64] FLAG: --eviction-hard="" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175438 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175440 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175443 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175449 2576 flags.go:64] FLAG: --eviction-soft="" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175451 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175454 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175457 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175461 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175464 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175466 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175469 2576 flags.go:64] FLAG: --feature-gates="" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175472 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175475 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 22:08:52.454351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175478 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175481 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175484 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175487 2576 flags.go:64] FLAG: --help="false" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175490 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175493 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175496 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175498 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175502 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175505 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175508 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175510 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175513 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175516 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175520 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175524 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175526 2576 flags.go:64] FLAG: --kube-reserved="" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175529 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175532 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175535 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175537 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175540 2576 flags.go:64] FLAG: --lock-file="" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175544 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175547 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 22:08:52.455203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175550 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175555 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175558 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175561 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175563 2576 flags.go:64] FLAG: --logging-format="text" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175566 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175569 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175572 2576 flags.go:64] FLAG: --manifest-url="" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175575 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175579 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175582 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175586 2576 flags.go:64] FLAG: --max-pods="110" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175589 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175592 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175594 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175597 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175600 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175602 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175605 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175612 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175614 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175617 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175622 2576 flags.go:64] FLAG: --pod-cidr="" Jul 09 22:08:52.455971 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175625 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175630 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175633 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175635 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175638 2576 flags.go:64] FLAG: --port="10250" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175641 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175644 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05dba4e8cb23a4b31" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175650 2576 flags.go:64] FLAG: --qos-reserved="" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175652 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175655 2576 flags.go:64] FLAG: --register-node="true" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175658 2576 flags.go:64] FLAG: --register-with-taints="" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175661 2576 flags.go:64] FLAG: --registry-burst="10" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175665 2576 flags.go:64] FLAG: --registry-qps="5" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175668 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175670 2576 flags.go:64] FLAG: --reserved-memory="" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175674 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175677 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175679 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175682 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175685 2576 flags.go:64] FLAG: --runonce="false" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175688 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175691 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175693 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175696 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175699 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175702 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 22:08:52.456708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175705 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175708 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175711 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175727 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175730 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175733 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175737 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175740 2576 flags.go:64] FLAG: --system-cgroups="" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175743 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175748 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175751 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175754 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175757 2576 flags.go:64] FLAG: --tls-min-version="" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175761 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175764 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175767 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175769 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175772 2576 flags.go:64] FLAG: --v="2" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175776 2576 flags.go:64] FLAG: --version="false" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175780 2576 flags.go:64] FLAG: --vmodule="" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175788 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.175791 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175903 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175906 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:08:52.457522 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175909 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175912 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175914 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175917 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175919 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175922 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175925 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175927 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175930 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175932 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175934 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175937 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175939 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175942 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175946 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175948 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175951 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175954 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175956 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175959 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:08:52.458269 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175961 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175965 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175967 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175970 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175973 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175975 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175978 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175980 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175983 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175986 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175988 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175991 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175994 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175996 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.175999 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176001 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176004 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176006 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176008 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:08:52.458923 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176011 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176013 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176016 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176020 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176023 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176026 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176029 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176033 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176036 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176039 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176042 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176044 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176047 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176049 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176053 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176055 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176058 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176060 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176063 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176065 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:08:52.460243 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176068 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176071 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176074 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176076 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176078 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176081 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176083 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176086 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176088 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176091 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176093 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176096 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176098 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176101 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176105 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176108 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176110 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176113 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176115 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176119 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:08:52.461206 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176122 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176124 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176127 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176129 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176132 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176134 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176138 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176141 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176143 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176146 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176150 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176153 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176156 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176159 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176162 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176165 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176168 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:08:52.462173 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.176170 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.176885 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.183954 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.183971 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.183990 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.184000 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184055 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184060 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184063 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184067 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184070 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184072 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184076 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184080 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:08:52.463113 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184085 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184088 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184091 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184094 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184097 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184099 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184102 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184104 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184108 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184111 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184113 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184116 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184118 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184121 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184124 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184127 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184130 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184132 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184135 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184138 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:08:52.463849 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184140 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184145 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184147 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184150 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184152 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184155 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184157 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184160 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184162 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184165 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184168 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184171 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184173 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184176 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184178 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184181 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184183 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184186 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184188 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184191 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:08:52.464772 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184193 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184196 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184198 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184201 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184203 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184206 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184211 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184214 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184217 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184220 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184222 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184225 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184227 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184230 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184233 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184235 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184238 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184240 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184243 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184245 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:08:52.465714 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184248 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184250 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184253 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184255 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184258 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184260 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184263 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184265 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184268 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184270 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184272 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184275 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184278 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184281 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184283 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184286 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184288 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184291 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184295 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184298 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184300 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:08:52.469207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184303 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184306 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184308 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184312 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184315 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184318 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184322 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184324 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184327 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184330 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.184336 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184446 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184450 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184453 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 22:08:52.470380 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184456 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184459 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184462 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184464 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184467 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184469 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184472 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184474 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184477 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184479 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184482 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184484 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184490 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184494 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184497 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184501 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184503 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184506 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184510 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 22:08:52.671740 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184513 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184516 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184519 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184522 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184525 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184527 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184536 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184539 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184541 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184544 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184547 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184549 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184552 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184554 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184557 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184559 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184562 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184564 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184567 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 22:08:52.673322 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184569 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184572 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184574 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184577 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184579 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184582 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184586 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184590 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184593 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184598 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184601 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184604 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184606 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184609 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184612 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184614 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184617 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184619 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184622 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184625 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 22:08:52.674292 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184638 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184640 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184643 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184646 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184648 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184651 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184653 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184656 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184658 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184661 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184663 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184666 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184668 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184671 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184674 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184676 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184679 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184681 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184685 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 22:08:52.675233 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184688 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184690 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184693 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184696 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184699 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184701 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184704 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184706 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184709 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184711 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184714 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184731 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184734 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184737 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184740 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184743 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184746 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184748 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 22:08:52.676195 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:52.184750 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.184755 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.184865 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.187269 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.188582 2576 server.go:1019] "Starting client certificate rotation" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.188678 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.188714 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.190861 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.191964 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.201379 2576 log.go:25] "Validated CRI v1 runtime API" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.204237 2576 log.go:25] "Validated CRI v1 image API" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.205789 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 22:08:52.677139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.208193 2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e6fdb82c-0c25-4fa6-b501-b8001135e833:/dev/nvme0n1p4 e9e1f10f-cdc7-4d50-aac6-82adf9861c92:/dev/nvme0n1p3] Jul 09 22:08:52.678075 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.208216 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 22:08:52.678305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.214354 2576 manager.go:216] Machine: {Timestamp:2026-07-09 22:08:52.213132406 +0000 UTC m=+0.311347586 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099387 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d63b6270dc6d46dcb6c7972ed304f SystemUUID:ec2d63b6-270d-c6d4-6dcb-6c7972ed304f BootID:48f1d1fb-064b-4a87-a751-0e8db71151c7 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dd:d3:60:db:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dd:d3:60:db:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:5d:1b:c3:d6:ad Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 22:08:52.678305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.214452 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 22:08:52.678305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.214529 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 22:08:52.678305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.217344 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 22:08:52.678305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.217375 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-244.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 22:08:52.678305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.218736 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 22:08:52.678305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.218748 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 22:08:52.678305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.218758 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.219372 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.221497 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.225559 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.225675 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.225701 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.225710 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.225741 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.225752 2576 kubelet.go:398] "Adding apiserver pod source" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.225761 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.226070 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.226085 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.226978 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.228226 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.230139 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.230166 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231589 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231607 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231616 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231624 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231637 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231645 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231656 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231664 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231672 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231681 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231695 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.231703 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.232378 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.232387 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.236071 2576 server.go:1268] "Started kubelet" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.236153 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.236297 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.236502 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.237671 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.238078 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.244326 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.244347 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.244488 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.244964 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.244981 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.245155 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.245271 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.245280 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 09 22:08:52.678806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.245639 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-244.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.245749 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.245769 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.245023 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-244.ec2.internal.18c0be8550e838a6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-244.ec2.internal,UID:ip-10-0-135-244.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-244.ec2.internal,},FirstTimestamp:2026-07-09 22:08:52.236040358 +0000 UTC m=+0.334255541,LastTimestamp:2026-07-09 22:08:52.236040358 +0000 UTC m=+0.334255541,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-244.ec2.internal,}" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.249688 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.249936 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.258355 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7djr8" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.259505 2576 factory.go:55] Registering systemd factory Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.259520 2576 factory.go:223] Registration of the systemd container factory successfully Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.259604 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.259806 2576 factory.go:153] Registering CRI-O factory Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.259819 2576 factory.go:223] Registration of the crio container factory successfully Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.259872 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.259894 2576 factory.go:103] Registering Raw factory Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.259910 2576 manager.go:1196] Started watching for new ooms in manager Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.260307 2576 manager.go:349] Starting recovery of all containers Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.268404 2576 manager.go:354] Recovery completed Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.268597 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7djr8" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.272243 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.273965 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.273987 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.273997 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.274569 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.274578 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.274596 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.274917 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.275471 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:08:52.681205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.275488 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.275497 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.276626 2576 policy_none.go:49] "None policy: Start" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.276639 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.276649 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.276695 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-244.ec2.internal.18c0be85532b1810 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-244.ec2.internal,UID:ip-10-0-135-244.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-244.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-244.ec2.internal,},FirstTimestamp:2026-07-09 22:08:52.27397736 +0000 UTC m=+0.372192541,LastTimestamp:2026-07-09 22:08:52.27397736 +0000 UTC m=+0.372192541,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-244.ec2.internal,}" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.278840 2576 policy_none.go:47] "Start" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.322356 2576 manager.go:337] "Starting Device Plugin manager" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.322605 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.322620 2576 server.go:87] "Starting device plugin registration server" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.322984 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.322998 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.323133 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.323218 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.323229 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.324078 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.324133 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.380665 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.381749 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.381783 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.381801 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.381832 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.385525 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.423776 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.425318 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.425349 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.425360 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.425390 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.442751 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.442773 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-244.ec2.internal\": node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.482837 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal"] Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.482901 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:08:52.682650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.484239 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.484331 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.484348 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.487113 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.487223 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.487250 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.488139 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.488149 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.488164 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.488173 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.488176 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.488188 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.490831 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.490857 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.491611 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientMemory" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.491638 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.491653 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeHasSufficientPID" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.492352 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.520545 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-244.ec2.internal\" not found" node="ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.525015 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-244.ec2.internal\" not found" node="ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.549084 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce1d60138de5db8ac5067e9c282bb219-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal\" (UID: \"ce1d60138de5db8ac5067e9c282bb219\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.549112 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1d60138de5db8ac5067e9c282bb219-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal\" (UID: \"ce1d60138de5db8ac5067e9c282bb219\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.549136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9528662dba682bac361080b2cce7b1c7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-244.ec2.internal\" (UID: \"9528662dba682bac361080b2cce7b1c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.593001 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:52.684308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.649643 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce1d60138de5db8ac5067e9c282bb219-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal\" (UID: \"ce1d60138de5db8ac5067e9c282bb219\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.685527 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.649684 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1d60138de5db8ac5067e9c282bb219-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal\" (UID: \"ce1d60138de5db8ac5067e9c282bb219\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.685527 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.649711 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9528662dba682bac361080b2cce7b1c7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-244.ec2.internal\" (UID: \"9528662dba682bac361080b2cce7b1c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.685527 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.649758 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce1d60138de5db8ac5067e9c282bb219-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal\" (UID: \"ce1d60138de5db8ac5067e9c282bb219\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.685527 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.649767 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce1d60138de5db8ac5067e9c282bb219-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal\" (UID: \"ce1d60138de5db8ac5067e9c282bb219\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.685527 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.649778 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9528662dba682bac361080b2cce7b1c7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-244.ec2.internal\" (UID: \"9528662dba682bac361080b2cce7b1c7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.867518 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.697111 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:52.867518 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.797230 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:52.867518 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.825299 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal" Jul 09 22:08:52.867518 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:52.828280 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" Jul 09 22:08:53.073423 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:52.898207 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:53.073423 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:53.001455 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:53.280554 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:53.102348 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:53.280554 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.105984 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:08:53.280554 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.190148 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 22:08:53.280554 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.190315 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 22:08:53.280554 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.190351 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 22:08:53.280554 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:53.203323 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:53.280554 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.272864 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 22:03:52 +0000 UTC" deadline="2028-01-22 01:23:26.05057099 +0000 UTC" Jul 09 22:08:53.280554 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.272892 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13467h14m32.777682469s" Jul 09 22:08:53.417323 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:53.303837 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:53.417323 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:53.404173 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:53.559814 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:53.504918 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-244.ec2.internal\" not found" Jul 09 22:08:53.559814 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.550021 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:08:53.707807 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.574109 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:08:53.707807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:53.581594 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9528662dba682bac361080b2cce7b1c7.slice/crio-582411c005e6dbd6bb0da05cd09e7f27afb290498ff9ef7222d995674b2eb312 WatchSource:0}: Error finding container 582411c005e6dbd6bb0da05cd09e7f27afb290498ff9ef7222d995674b2eb312: Status 404 returned error can't find the container with id 582411c005e6dbd6bb0da05cd09e7f27afb290498ff9ef7222d995674b2eb312 Jul 09 22:08:53.707807 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.587414 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:08:53.707807 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:53.594415 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1d60138de5db8ac5067e9c282bb219.slice/crio-0425107e5edcb7fd2e8a76d64d5b8726fc866786a7fc61eb15d2bc695901ec1b WatchSource:0}: Error finding container 0425107e5edcb7fd2e8a76d64d5b8726fc866786a7fc61eb15d2bc695901ec1b: Status 404 returned error can't find the container with id 0425107e5edcb7fd2e8a76d64d5b8726fc866786a7fc61eb15d2bc695901ec1b Jul 09 22:08:53.707807 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.646278 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal" Jul 09 22:08:53.707807 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.659810 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 22:08:53.707807 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.660788 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" Jul 09 22:08:53.707807 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:53.675510 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.228346 2576 apiserver.go:52] "Watching apiserver" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.242214 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.243554 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-65pnm","openshift-image-registry/node-ca-lj48t","openshift-multus/multus-additional-cni-plugins-5jqh2","openshift-network-diagnostics/network-check-target-bqdc5","kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal","openshift-multus/multus-zxpng","openshift-multus/network-metrics-daemon-6969j","openshift-network-operator/iptables-alerter-j7999","openshift-ovn-kubernetes/ovnkube-node-7clnl","kube-system/global-pull-secret-syncer-vm5vl","kube-system/konnectivity-agent-w75wq","openshift-cluster-node-tuning-operator/tuned-xd2gj"] Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.246085 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.246275 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.246353 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.248621 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.251057 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.253500 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.256117 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.258886 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.260961 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-lib-modules\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.260990 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-tuned\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261016 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261041 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87dh9\" (UniqueName: \"kubernetes.io/projected/173926ea-7469-4f39-a90b-5ddaf5ec576c-kube-api-access-87dh9\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261147 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-modprobe-d\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261183 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-kubernetes\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.281902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261215 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-var-lib-kubelet\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261237 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-host\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261269 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-sysctl-d\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261296 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-sys\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261318 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-tmp\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261341 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-sysctl-conf\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261379 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-run\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261399 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kstcl\" (UniqueName: \"kubernetes.io/projected/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-kube-api-access-kstcl\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261563 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-sysconfig\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261634 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.261647 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-systemd\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.265500 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.268193 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.271672 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.271767 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.273249 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.273436 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cc2kh\"" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.273480 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.273611 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-swtc5\"" Jul 09 22:08:54.347733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.273628 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 22:08:54.349347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.273755 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 22:08:54.349347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.273764 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 22:08:54.349347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.273984 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 22:08:54.349347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.274365 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:08:54.349347 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.274433 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:08:54.349347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.277082 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.349347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.346746 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 22:08:54.361939 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.361901 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-run\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.362042 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.361954 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.362042 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.361984 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/976f8a42-342d-4902-8d9a-1c6753f67d83-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.362042 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-systemd\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.362042 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362038 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-lib-modules\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.362264 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362063 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgxhv\" (UniqueName: \"kubernetes.io/projected/7e53afda-97f8-4484-9c7c-b747bde49e8f-kube-api-access-hgxhv\") pod \"node-ca-lj48t\" (UID: \"7e53afda-97f8-4484-9c7c-b747bde49e8f\") " pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.362264 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362088 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-log-socket\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.362264 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-etc-kubernetes\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.362264 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362152 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-host\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.362264 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362176 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e53afda-97f8-4484-9c7c-b747bde49e8f-serviceca\") pod \"node-ca-lj48t\" (UID: \"7e53afda-97f8-4484-9c7c-b747bde49e8f\") " pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.362264 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362202 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.362264 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362227 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-var-lib-kubelet\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.362264 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362255 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-run-multus-certs\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362281 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-tmp\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362307 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-socket-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362332 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-run-netns\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362374 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-run-systemd\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362402 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-cni-bin\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-var-lib-cni-bin\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362463 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-system-cni-dir\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362497 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/976f8a42-342d-4902-8d9a-1c6753f67d83-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362524 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b276e595-42b4-452f-8c2b-703a2f894f07-dbus\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-cni-dir\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362574 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-conf-dir\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362601 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87dh9\" (UniqueName: \"kubernetes.io/projected/173926ea-7469-4f39-a90b-5ddaf5ec576c-kube-api-access-87dh9\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362629 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dff8d8d0-93d0-40a2-8061-9fb8f8903316-hosts-file\") pod \"node-resolver-65pnm\" (UID: \"dff8d8d0-93d0-40a2-8061-9fb8f8903316\") " pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.362657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362656 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-kubelet\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362681 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-run-openvswitch\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362720 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362748 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-modprobe-d\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362781 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-socket-dir-parent\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362824 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-run-k8s-cni-cncf-io\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362863 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-sys\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362889 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-registration-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362919 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-system-cni-dir\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362945 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-var-lib-cni-multus\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.362981 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-hostroot\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363009 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/210ead62-927e-4f84-82b3-d26033a4219a-iptables-alerter-script\") pod \"iptables-alerter-j7999\" (UID: \"210ead62-927e-4f84-82b3-d26033a4219a\") " pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363037 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/210ead62-927e-4f84-82b3-d26033a4219a-host-slash\") pod \"iptables-alerter-j7999\" (UID: \"210ead62-927e-4f84-82b3-d26033a4219a\") " pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-sysctl-conf\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.363368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363153 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kstcl\" (UniqueName: \"kubernetes.io/projected/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-kube-api-access-kstcl\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363180 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363236 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363264 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ddpv\" (UniqueName: \"kubernetes.io/projected/dff8d8d0-93d0-40a2-8061-9fb8f8903316-kube-api-access-2ddpv\") pod \"node-resolver-65pnm\" (UID: \"dff8d8d0-93d0-40a2-8061-9fb8f8903316\") " pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363297 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-slash\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363322 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-cni-netd\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fea577f9-5cea-46b9-9591-c88c1549dea5-ovn-node-metrics-cert\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363389 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-tuned\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363420 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-sys-fs\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363449 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5-konnectivity-ca\") pod \"konnectivity-agent-w75wq\" (UID: \"4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5\") " pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363475 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-etc-openvswitch\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363501 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-node-log\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363527 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fea577f9-5cea-46b9-9591-c88c1549dea5-ovnkube-config\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363556 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-kubernetes\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.364074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363584 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-var-lib-openvswitch\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363611 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fea577f9-5cea-46b9-9591-c88c1549dea5-env-overrides\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-run-netns\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363663 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-daemon-config\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363689 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v5ns\" (UniqueName: \"kubernetes.io/projected/210ead62-927e-4f84-82b3-d26033a4219a-kube-api-access-7v5ns\") pod \"iptables-alerter-j7999\" (UID: \"210ead62-927e-4f84-82b3-d26033a4219a\") " pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363728 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-sysctl-d\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363755 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b276e595-42b4-452f-8c2b-703a2f894f07-kubelet-config\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363782 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5-agent-certs\") pod \"konnectivity-agent-w75wq\" (UID: \"4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5\") " pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363840 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/976f8a42-342d-4902-8d9a-1c6753f67d83-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363870 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-device-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363924 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363952 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fea577f9-5cea-46b9-9591-c88c1549dea5-ovnkube-script-lib\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.363988 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-cnibin\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.364721 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364021 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-sysconfig\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364050 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364081 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e53afda-97f8-4484-9c7c-b747bde49e8f-host\") pod \"node-ca-lj48t\" (UID: \"7e53afda-97f8-4484-9c7c-b747bde49e8f\") " pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364111 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq9qc\" (UniqueName: \"kubernetes.io/projected/976f8a42-342d-4902-8d9a-1c6753f67d83-kube-api-access-nq9qc\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364140 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-run-ovn\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364170 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-cni-binary-copy\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364198 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvvl5\" (UniqueName: \"kubernetes.io/projected/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-kube-api-access-rvvl5\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364240 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-var-lib-kubelet\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364269 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-os-release\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364297 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff8d8d0-93d0-40a2-8061-9fb8f8903316-tmp-dir\") pod \"node-resolver-65pnm\" (UID: \"dff8d8d0-93d0-40a2-8061-9fb8f8903316\") " pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364325 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-systemd-units\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364356 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h5t2\" (UniqueName: \"kubernetes.io/projected/fea577f9-5cea-46b9-9591-c88c1549dea5-kube-api-access-6h5t2\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364386 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-cnibin\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364417 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7vxh\" (UniqueName: \"kubernetes.io/projected/09ffcc38-f771-46ba-acab-e1c4a45eedda-kube-api-access-f7vxh\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.365416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364445 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-os-release\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.366141 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364636 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-run\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366141 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364725 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-systemd\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366141 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364851 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-lib-modules\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366141 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.364923 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-host\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366141 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.365329 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 22:08:54.366141 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.366061 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-sysctl-d\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.366140 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-sys\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.366201 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-kubernetes\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.366300 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-modprobe-d\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.366308 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-sysctl-conf\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366916 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.366700 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-var-lib-kubelet\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366916 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.366700 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-sysconfig\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.366916 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.366781 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:08:54.366916 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.366874 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:08:54.866850915 +0000 UTC m=+2.965066112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:08:54.374015 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.373962 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-tmp\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.375509 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.375484 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nw9v6\"" Jul 09 22:08:54.375881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.375859 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 22:08:54.376235 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.376051 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lbphc\"" Jul 09 22:08:54.376235 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.376062 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 22:08:54.376235 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.376064 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 22:08:54.376235 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.376084 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wsjtt\"" Jul 09 22:08:54.377263 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.377241 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-etc-tuned\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.378574 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.377790 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bw9k2\"" Jul 09 22:08:54.378574 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378053 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 22:08:54.378574 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378132 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 22:08:54.378574 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378304 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 22:08:54.378574 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378331 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 22:08:54.378574 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378339 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 22:08:54.378574 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378466 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 22:08:54.379011 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378628 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 22:08:54.379011 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378634 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s8vdc\"" Jul 09 22:08:54.379011 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378767 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 22:08:54.379011 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378797 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 22:08:54.379011 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378811 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 22:08:54.379011 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378821 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 22:08:54.379011 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378955 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p7n2t\"" Jul 09 22:08:54.379011 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.378991 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 22:08:54.379494 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.379113 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 22:08:54.379494 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.379148 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 22:08:54.379494 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.379199 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 22:08:54.380162 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.380143 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 22:08:54.380162 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.380156 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 22:08:54.380365 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.380346 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 22:08:54.380528 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.380505 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b9cfx\"" Jul 09 22:08:54.385066 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.385040 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kstcl\" (UniqueName: \"kubernetes.io/projected/1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4-kube-api-access-kstcl\") pod \"tuned-xd2gj\" (UID: \"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4\") " pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.385494 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.385467 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 22:08:54.385576 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.385538 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9zp2s" Jul 09 22:08:54.386572 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.386549 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87dh9\" (UniqueName: \"kubernetes.io/projected/173926ea-7469-4f39-a90b-5ddaf5ec576c-kube-api-access-87dh9\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:54.396954 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.396568 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" event={"ID":"ce1d60138de5db8ac5067e9c282bb219","Type":"ContainerStarted","Data":"0425107e5edcb7fd2e8a76d64d5b8726fc866786a7fc61eb15d2bc695901ec1b"} Jul 09 22:08:54.400808 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.400767 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal" event={"ID":"9528662dba682bac361080b2cce7b1c7","Type":"ContainerStarted","Data":"582411c005e6dbd6bb0da05cd09e7f27afb290498ff9ef7222d995674b2eb312"} Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.465817 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/976f8a42-342d-4902-8d9a-1c6753f67d83-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.465866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-device-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.465951 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-device-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.465991 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466020 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466045 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fea577f9-5cea-46b9-9591-c88c1549dea5-ovnkube-script-lib\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466070 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-cnibin\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466112 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e53afda-97f8-4484-9c7c-b747bde49e8f-host\") pod \"node-ca-lj48t\" (UID: \"7e53afda-97f8-4484-9c7c-b747bde49e8f\") " pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466139 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq9qc\" (UniqueName: \"kubernetes.io/projected/976f8a42-342d-4902-8d9a-1c6753f67d83-kube-api-access-nq9qc\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466165 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-run-ovn\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466198 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-cni-binary-copy\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466225 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvvl5\" (UniqueName: \"kubernetes.io/projected/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-kube-api-access-rvvl5\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-os-release\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.467077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466277 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff8d8d0-93d0-40a2-8061-9fb8f8903316-tmp-dir\") pod \"node-resolver-65pnm\" (UID: \"dff8d8d0-93d0-40a2-8061-9fb8f8903316\") " pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466304 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-systemd-units\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466329 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h5t2\" (UniqueName: \"kubernetes.io/projected/fea577f9-5cea-46b9-9591-c88c1549dea5-kube-api-access-6h5t2\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466355 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-cnibin\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466381 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7vxh\" (UniqueName: \"kubernetes.io/projected/09ffcc38-f771-46ba-acab-e1c4a45eedda-kube-api-access-f7vxh\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466406 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-os-release\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466414 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-run-ovn\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466434 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466478 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/976f8a42-342d-4902-8d9a-1c6753f67d83-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466493 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466503 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/976f8a42-342d-4902-8d9a-1c6753f67d83-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466516 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgxhv\" (UniqueName: \"kubernetes.io/projected/7e53afda-97f8-4484-9c7c-b747bde49e8f-kube-api-access-hgxhv\") pod \"node-ca-lj48t\" (UID: \"7e53afda-97f8-4484-9c7c-b747bde49e8f\") " pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466545 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-log-socket\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.466550 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466570 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-etc-kubernetes\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.468007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466556 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-systemd-units\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.466607 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret podName:b276e595-42b4-452f-8c2b-703a2f894f07 nodeName:}" failed. No retries permitted until 2026-07-09 22:08:54.966590053 +0000 UTC m=+3.064805229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret") pod "global-pull-secret-syncer-vm5vl" (UID: "b276e595-42b4-452f-8c2b-703a2f894f07") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466480 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466654 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-log-socket\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466695 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-etc-kubernetes\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466743 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-cnibin\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466783 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-cnibin\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466875 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-os-release\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.466925 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e53afda-97f8-4484-9c7c-b747bde49e8f-host\") pod \"node-ca-lj48t\" (UID: \"7e53afda-97f8-4484-9c7c-b747bde49e8f\") " pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467190 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-cni-binary-copy\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467231 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e53afda-97f8-4484-9c7c-b747bde49e8f-serviceca\") pod \"node-ca-lj48t\" (UID: \"7e53afda-97f8-4484-9c7c-b747bde49e8f\") " pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467263 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467291 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-var-lib-kubelet\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-run-multus-certs\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.468746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467344 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-socket-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467369 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-run-netns\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467392 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-run-systemd\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-cni-bin\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467441 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-var-lib-cni-bin\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467449 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/976f8a42-342d-4902-8d9a-1c6753f67d83-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467464 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fea577f9-5cea-46b9-9591-c88c1549dea5-ovnkube-script-lib\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467468 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-system-cni-dir\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467500 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-system-cni-dir\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467515 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/976f8a42-342d-4902-8d9a-1c6753f67d83-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467524 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467534 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/976f8a42-342d-4902-8d9a-1c6753f67d83-os-release\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467544 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b276e595-42b4-452f-8c2b-703a2f894f07-dbus\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467554 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-run-systemd\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.469481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467582 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-cni-bin\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467610 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-cni-dir\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467635 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-socket-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467638 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-conf-dir\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467670 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-conf-dir\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467679 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dff8d8d0-93d0-40a2-8061-9fb8f8903316-hosts-file\") pod \"node-resolver-65pnm\" (UID: \"dff8d8d0-93d0-40a2-8061-9fb8f8903316\") " pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467528 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-var-lib-kubelet\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467709 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b276e595-42b4-452f-8c2b-703a2f894f07-dbus\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467709 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-kubelet\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467764 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-var-lib-cni-bin\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467760 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-run-multus-certs\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467823 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-run-openvswitch\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467833 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-run-netns\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467858 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-socket-dir-parent\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467895 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-run-openvswitch\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467914 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-run-k8s-cni-cncf-io\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467943 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-registration-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467951 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dff8d8d0-93d0-40a2-8061-9fb8f8903316-hosts-file\") pod \"node-resolver-65pnm\" (UID: \"dff8d8d0-93d0-40a2-8061-9fb8f8903316\") " pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467969 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-system-cni-dir\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.467994 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-var-lib-cni-multus\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468020 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-hostroot\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468050 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/210ead62-927e-4f84-82b3-d26033a4219a-iptables-alerter-script\") pod \"iptables-alerter-j7999\" (UID: \"210ead62-927e-4f84-82b3-d26033a4219a\") " pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/210ead62-927e-4f84-82b3-d26033a4219a-host-slash\") pod \"iptables-alerter-j7999\" (UID: \"210ead62-927e-4f84-82b3-d26033a4219a\") " pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468104 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468128 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468156 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddpv\" (UniqueName: \"kubernetes.io/projected/dff8d8d0-93d0-40a2-8061-9fb8f8903316-kube-api-access-2ddpv\") pod \"node-resolver-65pnm\" (UID: \"dff8d8d0-93d0-40a2-8061-9fb8f8903316\") " pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468166 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-socket-dir-parent\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468181 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-slash\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.470978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468208 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-cni-netd\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468214 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-run-k8s-cni-cncf-io\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468214 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/976f8a42-342d-4902-8d9a-1c6753f67d83-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468232 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fea577f9-5cea-46b9-9591-c88c1549dea5-ovn-node-metrics-cert\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468251 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e53afda-97f8-4484-9c7c-b747bde49e8f-serviceca\") pod \"node-ca-lj48t\" (UID: \"7e53afda-97f8-4484-9c7c-b747bde49e8f\") " pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468263 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-registration-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468274 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-system-cni-dir\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468263 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-sys-fs\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468051 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-kubelet\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5-konnectivity-ca\") pod \"konnectivity-agent-w75wq\" (UID: \"4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5\") " pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468314 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dff8d8d0-93d0-40a2-8061-9fb8f8903316-tmp-dir\") pod \"node-resolver-65pnm\" (UID: \"dff8d8d0-93d0-40a2-8061-9fb8f8903316\") " pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468317 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-sys-fs\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468343 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/210ead62-927e-4f84-82b3-d26033a4219a-host-slash\") pod \"iptables-alerter-j7999\" (UID: \"210ead62-927e-4f84-82b3-d26033a4219a\") " pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468023 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-cni-dir\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468367 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-etc-openvswitch\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.471646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468374 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-hostroot\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468396 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-node-log\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468418 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468424 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fea577f9-5cea-46b9-9591-c88c1549dea5-ovnkube-config\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468433 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-cni-netd\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468438 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-etc-openvswitch\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468395 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-host-slash\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468465 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09ffcc38-f771-46ba-acab-e1c4a45eedda-etc-selinux\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468475 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-node-log\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468453 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-var-lib-openvswitch\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468513 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fea577f9-5cea-46b9-9591-c88c1549dea5-var-lib-openvswitch\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468505 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-var-lib-cni-multus\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468536 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fea577f9-5cea-46b9-9591-c88c1549dea5-env-overrides\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468568 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-run-netns\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468599 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-daemon-config\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.472418 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v5ns\" (UniqueName: \"kubernetes.io/projected/210ead62-927e-4f84-82b3-d26033a4219a-kube-api-access-7v5ns\") pod \"iptables-alerter-j7999\" (UID: \"210ead62-927e-4f84-82b3-d26033a4219a\") " pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468650 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b276e595-42b4-452f-8c2b-703a2f894f07-kubelet-config\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.468676 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5-agent-certs\") pod \"konnectivity-agent-w75wq\" (UID: \"4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5\") " pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.469012 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5-konnectivity-ca\") pod \"konnectivity-agent-w75wq\" (UID: \"4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5\") " pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.469021 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fea577f9-5cea-46b9-9591-c88c1549dea5-ovnkube-config\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.469025 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fea577f9-5cea-46b9-9591-c88c1549dea5-env-overrides\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.469089 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-host-run-netns\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.469144 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b276e595-42b4-452f-8c2b-703a2f894f07-kubelet-config\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.469454 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-multus-daemon-config\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.469564 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/210ead62-927e-4f84-82b3-d26033a4219a-iptables-alerter-script\") pod \"iptables-alerter-j7999\" (UID: \"210ead62-927e-4f84-82b3-d26033a4219a\") " pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.473187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.471929 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fea577f9-5cea-46b9-9591-c88c1549dea5-ovn-node-metrics-cert\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.473698 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.473573 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5-agent-certs\") pod \"konnectivity-agent-w75wq\" (UID: \"4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5\") " pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:08:54.487343 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.487303 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgxhv\" (UniqueName: \"kubernetes.io/projected/7e53afda-97f8-4484-9c7c-b747bde49e8f-kube-api-access-hgxhv\") pod \"node-ca-lj48t\" (UID: \"7e53afda-97f8-4484-9c7c-b747bde49e8f\") " pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.487999 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.487976 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:08:54.487999 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.488000 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:08:54.488183 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.488013 2576 projected.go:196] Error preparing data for projected volume kube-api-access-fbkb5 for pod openshift-network-diagnostics/network-check-target-bqdc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:08:54.488183 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.488068 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5 podName:1a1bfaf6-12e3-4a89-b415-a5de9061cbba nodeName:}" failed. No retries permitted until 2026-07-09 22:08:54.98805324 +0000 UTC m=+3.086268410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fbkb5" (UniqueName: "kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5") pod "network-check-target-bqdc5" (UID: "1a1bfaf6-12e3-4a89-b415-a5de9061cbba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:08:54.488906 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.488882 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq9qc\" (UniqueName: \"kubernetes.io/projected/976f8a42-342d-4902-8d9a-1c6753f67d83-kube-api-access-nq9qc\") pod \"multus-additional-cni-plugins-5jqh2\" (UID: \"976f8a42-342d-4902-8d9a-1c6753f67d83\") " pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.489355 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.489331 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7vxh\" (UniqueName: \"kubernetes.io/projected/09ffcc38-f771-46ba-acab-e1c4a45eedda-kube-api-access-f7vxh\") pod \"aws-ebs-csi-driver-node-jlkmx\" (UID: \"09ffcc38-f771-46ba-acab-e1c4a45eedda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.490758 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.490731 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v5ns\" (UniqueName: \"kubernetes.io/projected/210ead62-927e-4f84-82b3-d26033a4219a-kube-api-access-7v5ns\") pod \"iptables-alerter-j7999\" (UID: \"210ead62-927e-4f84-82b3-d26033a4219a\") " pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.491137 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.491114 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvvl5\" (UniqueName: \"kubernetes.io/projected/bd4e48cc-2d92-4a9c-a23f-e25fe939fd87-kube-api-access-rvvl5\") pod \"multus-zxpng\" (UID: \"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87\") " pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.491863 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.491839 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h5t2\" (UniqueName: \"kubernetes.io/projected/fea577f9-5cea-46b9-9591-c88c1549dea5-kube-api-access-6h5t2\") pod \"ovnkube-node-7clnl\" (UID: \"fea577f9-5cea-46b9-9591-c88c1549dea5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.491939 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.491905 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ddpv\" (UniqueName: \"kubernetes.io/projected/dff8d8d0-93d0-40a2-8061-9fb8f8903316-kube-api-access-2ddpv\") pod \"node-resolver-65pnm\" (UID: \"dff8d8d0-93d0-40a2-8061-9fb8f8903316\") " pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.560191 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.560153 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" Jul 09 22:08:54.570707 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.570681 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" Jul 09 22:08:54.574068 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:54.574038 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ba084a3_ee75_4a21_b72c_5f7fe4e74fd4.slice/crio-82c0daf8d869d77728f5e48e885d3f0341819607987e2d00f4a33942c7f97a5c WatchSource:0}: Error finding container 82c0daf8d869d77728f5e48e885d3f0341819607987e2d00f4a33942c7f97a5c: Status 404 returned error can't find the container with id 82c0daf8d869d77728f5e48e885d3f0341819607987e2d00f4a33942c7f97a5c Jul 09 22:08:54.581080 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.580892 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:08:54.583341 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:54.583313 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod976f8a42_342d_4902_8d9a_1c6753f67d83.slice/crio-9bdc4f194a9ea5b0b005bd1cbb64aa1cf2cc6bb7fc2670d75fb191e84ddccd68 WatchSource:0}: Error finding container 9bdc4f194a9ea5b0b005bd1cbb64aa1cf2cc6bb7fc2670d75fb191e84ddccd68: Status 404 returned error can't find the container with id 9bdc4f194a9ea5b0b005bd1cbb64aa1cf2cc6bb7fc2670d75fb191e84ddccd68 Jul 09 22:08:54.586944 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.586916 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:08:54.599934 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.599671 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" Jul 09 22:08:54.601265 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:54.601220 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea577f9_5cea_46b9_9591_c88c1549dea5.slice/crio-3dd21b5801c91d6b568cd5f5ed00f0da3715017426d329f507e637fa648fa15c WatchSource:0}: Error finding container 3dd21b5801c91d6b568cd5f5ed00f0da3715017426d329f507e637fa648fa15c: Status 404 returned error can't find the container with id 3dd21b5801c91d6b568cd5f5ed00f0da3715017426d329f507e637fa648fa15c Jul 09 22:08:54.606827 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.606768 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-65pnm" Jul 09 22:08:54.613419 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:54.613391 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09ffcc38_f771_46ba_acab_e1c4a45eedda.slice/crio-481d8f5574319487d74b4fe4f6a744a22fc158913c3fe04ada47856c58587c19 WatchSource:0}: Error finding container 481d8f5574319487d74b4fe4f6a744a22fc158913c3fe04ada47856c58587c19: Status 404 returned error can't find the container with id 481d8f5574319487d74b4fe4f6a744a22fc158913c3fe04ada47856c58587c19 Jul 09 22:08:54.615164 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.615141 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zxpng" Jul 09 22:08:54.619331 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:54.619305 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff8d8d0_93d0_40a2_8061_9fb8f8903316.slice/crio-03cd2574c2b2be9ca448fbf8186a872dae937d31a9b53d843e8e3ce8719981b9 WatchSource:0}: Error finding container 03cd2574c2b2be9ca448fbf8186a872dae937d31a9b53d843e8e3ce8719981b9: Status 404 returned error can't find the container with id 03cd2574c2b2be9ca448fbf8186a872dae937d31a9b53d843e8e3ce8719981b9 Jul 09 22:08:54.626803 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:54.626776 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd4e48cc_2d92_4a9c_a23f_e25fe939fd87.slice/crio-49b3f224a5671a5256fb9b5fbe9ff1f32f7fdf31295dae968a3a9c1de5f925da WatchSource:0}: Error finding container 49b3f224a5671a5256fb9b5fbe9ff1f32f7fdf31295dae968a3a9c1de5f925da: Status 404 returned error can't find the container with id 49b3f224a5671a5256fb9b5fbe9ff1f32f7fdf31295dae968a3a9c1de5f925da Jul 09 22:08:54.626983 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.626958 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:08:54.672735 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.672697 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j7999" Jul 09 22:08:54.683918 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:54.683885 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210ead62_927e_4f84_82b3_d26033a4219a.slice/crio-12dbc4df83bb01d7d5e3945d2874a3bc7f81cb6a3ed803706ad2c7a5a3413b06 WatchSource:0}: Error finding container 12dbc4df83bb01d7d5e3945d2874a3bc7f81cb6a3ed803706ad2c7a5a3413b06: Status 404 returned error can't find the container with id 12dbc4df83bb01d7d5e3945d2874a3bc7f81cb6a3ed803706ad2c7a5a3413b06 Jul 09 22:08:54.701133 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.701089 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lj48t" Jul 09 22:08:54.871628 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.871530 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:54.871805 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.871656 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:08:54.871805 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.871718 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:08:55.871702855 +0000 UTC m=+3.969918025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:08:54.972200 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:54.972152 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:54.972386 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.972336 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:08:54.972438 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:54.972412 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret podName:b276e595-42b4-452f-8c2b-703a2f894f07 nodeName:}" failed. No retries permitted until 2026-07-09 22:08:55.972393026 +0000 UTC m=+4.070608218 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret") pod "global-pull-secret-syncer-vm5vl" (UID: "b276e595-42b4-452f-8c2b-703a2f894f07") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:08:55.072998 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.072961 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:08:55.073149 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:55.073113 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:08:55.073149 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:55.073129 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:08:55.073149 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:55.073139 2576 projected.go:196] Error preparing data for projected volume kube-api-access-fbkb5 for pod openshift-network-diagnostics/network-check-target-bqdc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:08:55.073258 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:55.073197 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5 podName:1a1bfaf6-12e3-4a89-b415-a5de9061cbba nodeName:}" failed. No retries permitted until 2026-07-09 22:08:56.073180532 +0000 UTC m=+4.171395720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbkb5" (UniqueName: "kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5") pod "network-check-target-bqdc5" (UID: "1a1bfaf6-12e3-4a89-b415-a5de9061cbba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:08:55.393904 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.393828 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 22:03:54 +0000 UTC" deadline="2028-02-20 13:48:45.037703075 +0000 UTC" Jul 09 22:08:55.393904 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.393898 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14175h39m49.643810383s" Jul 09 22:08:55.404228 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.404114 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j7999" event={"ID":"210ead62-927e-4f84-82b3-d26033a4219a","Type":"ContainerStarted","Data":"12dbc4df83bb01d7d5e3945d2874a3bc7f81cb6a3ed803706ad2c7a5a3413b06"} Jul 09 22:08:55.406282 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.406252 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zxpng" event={"ID":"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87","Type":"ContainerStarted","Data":"49b3f224a5671a5256fb9b5fbe9ff1f32f7fdf31295dae968a3a9c1de5f925da"} Jul 09 22:08:55.407596 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.407539 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-65pnm" event={"ID":"dff8d8d0-93d0-40a2-8061-9fb8f8903316","Type":"ContainerStarted","Data":"03cd2574c2b2be9ca448fbf8186a872dae937d31a9b53d843e8e3ce8719981b9"} Jul 09 22:08:55.410748 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.410673 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" event={"ID":"09ffcc38-f771-46ba-acab-e1c4a45eedda","Type":"ContainerStarted","Data":"481d8f5574319487d74b4fe4f6a744a22fc158913c3fe04ada47856c58587c19"} Jul 09 22:08:55.414056 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.414024 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" event={"ID":"fea577f9-5cea-46b9-9591-c88c1549dea5","Type":"ContainerStarted","Data":"3dd21b5801c91d6b568cd5f5ed00f0da3715017426d329f507e637fa648fa15c"} Jul 09 22:08:55.415964 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.415938 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w75wq" event={"ID":"4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5","Type":"ContainerStarted","Data":"196daa94bdbecac9c89281b56aa8916a2ce36637535fa449321486baa087f5dd"} Jul 09 22:08:55.417216 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.417170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" event={"ID":"976f8a42-342d-4902-8d9a-1c6753f67d83","Type":"ContainerStarted","Data":"9bdc4f194a9ea5b0b005bd1cbb64aa1cf2cc6bb7fc2670d75fb191e84ddccd68"} Jul 09 22:08:55.418939 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.418892 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" event={"ID":"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4","Type":"ContainerStarted","Data":"82c0daf8d869d77728f5e48e885d3f0341819607987e2d00f4a33942c7f97a5c"} Jul 09 22:08:55.880196 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.880103 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:55.880386 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:55.880263 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:08:55.880386 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:55.880344 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:08:57.880324258 +0000 UTC m=+5.978539434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:08:55.963870 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.963547 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 22:08:55.981351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:55.980764 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:55.981351 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:55.980944 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:08:55.981351 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:55.981012 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret podName:b276e595-42b4-452f-8c2b-703a2f894f07 nodeName:}" failed. No retries permitted until 2026-07-09 22:08:57.98099196 +0000 UTC m=+6.079207151 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret") pod "global-pull-secret-syncer-vm5vl" (UID: "b276e595-42b4-452f-8c2b-703a2f894f07") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:08:56.082126 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:56.081373 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:08:56.082126 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:56.081602 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:08:56.082126 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:56.081624 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:08:56.082126 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:56.081639 2576 projected.go:196] Error preparing data for projected volume kube-api-access-fbkb5 for pod openshift-network-diagnostics/network-check-target-bqdc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:08:56.082126 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:56.081711 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5 podName:1a1bfaf6-12e3-4a89-b415-a5de9061cbba nodeName:}" failed. No retries permitted until 2026-07-09 22:08:58.081693004 +0000 UTC m=+6.179908199 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbkb5" (UniqueName: "kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5") pod "network-check-target-bqdc5" (UID: "1a1bfaf6-12e3-4a89-b415-a5de9061cbba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:08:56.383944 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:56.382919 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:56.383944 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:56.383059 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:08:56.383944 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:56.383494 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:08:56.383944 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:56.383587 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:08:56.383944 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:56.383937 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:56.384318 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:56.384064 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:08:56.394371 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:56.394307 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 22:03:54 +0000 UTC" deadline="2028-01-02 04:38:24.423670843 +0000 UTC" Jul 09 22:08:56.394371 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:56.394340 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12990h29m28.029335706s" Jul 09 22:08:56.442936 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:08:56.442905 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e53afda_97f8_4484_9c7c_b747bde49e8f.slice/crio-2fe6f63f68831591c840eed12fe7bd93395cc58427bfa476ef02dacd3a10865d WatchSource:0}: Error finding container 2fe6f63f68831591c840eed12fe7bd93395cc58427bfa476ef02dacd3a10865d: Status 404 returned error can't find the container with id 2fe6f63f68831591c840eed12fe7bd93395cc58427bfa476ef02dacd3a10865d Jul 09 22:08:57.438011 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:57.437568 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" event={"ID":"ce1d60138de5db8ac5067e9c282bb219","Type":"ContainerStarted","Data":"448f0f4ebd7506ea92288b258d735d0ba2621e58ad8f4c26a95f6298aa18f13a"} Jul 09 22:08:57.444435 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:57.444405 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal" event={"ID":"9528662dba682bac361080b2cce7b1c7","Type":"ContainerStarted","Data":"59302b8ce12ac0bed1fbb7a6e6594f9b082c6328d3b43dc22a74cf27fa9cd39f"} Jul 09 22:08:57.450159 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:57.450104 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lj48t" event={"ID":"7e53afda-97f8-4484-9c7c-b747bde49e8f","Type":"ContainerStarted","Data":"2fe6f63f68831591c840eed12fe7bd93395cc58427bfa476ef02dacd3a10865d"} Jul 09 22:08:57.900243 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:57.899512 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:57.900243 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:57.899697 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:08:57.900243 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:57.899781 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:01.8997624 +0000 UTC m=+9.997977572 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:08:58.001262 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:58.000613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:58.001262 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:58.000837 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:08:58.001262 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:58.000904 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret podName:b276e595-42b4-452f-8c2b-703a2f894f07 nodeName:}" failed. No retries permitted until 2026-07-09 22:09:02.000883891 +0000 UTC m=+10.099099077 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret") pod "global-pull-secret-syncer-vm5vl" (UID: "b276e595-42b4-452f-8c2b-703a2f894f07") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:08:58.101926 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:58.101881 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:08:58.102109 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:58.102086 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:08:58.102109 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:58.102104 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:08:58.102202 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:58.102117 2576 projected.go:196] Error preparing data for projected volume kube-api-access-fbkb5 for pod openshift-network-diagnostics/network-check-target-bqdc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:08:58.102202 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:58.102175 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5 podName:1a1bfaf6-12e3-4a89-b415-a5de9061cbba nodeName:}" failed. No retries permitted until 2026-07-09 22:09:02.102158192 +0000 UTC m=+10.200373364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbkb5" (UniqueName: "kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5") pod "network-check-target-bqdc5" (UID: "1a1bfaf6-12e3-4a89-b415-a5de9061cbba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:08:58.384817 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:58.384282 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:08:58.384817 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:58.384422 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:08:58.384817 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:58.384796 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:08:58.385076 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:58.384897 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:08:58.385076 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:08:58.384950 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:08:58.385076 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:08:58.385044 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:00.386431 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:00.386386 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:00.386909 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:00.386386 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:00.386909 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:00.386512 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:00.386909 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:00.386554 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:00.386909 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:00.386664 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:00.386909 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:00.386754 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:00.460230 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:00.460195 2576 generic.go:356] "Generic (PLEG): container finished" podID="ce1d60138de5db8ac5067e9c282bb219" containerID="448f0f4ebd7506ea92288b258d735d0ba2621e58ad8f4c26a95f6298aa18f13a" exitCode=0 Jul 09 22:09:00.460420 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:00.460253 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" event={"ID":"ce1d60138de5db8ac5067e9c282bb219","Type":"ContainerDied","Data":"448f0f4ebd7506ea92288b258d735d0ba2621e58ad8f4c26a95f6298aa18f13a"} Jul 09 22:09:00.481972 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:00.481771 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-244.ec2.internal" podStartSLOduration=7.481755338 podStartE2EDuration="7.481755338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:08:57.478236678 +0000 UTC m=+5.576451869" watchObservedRunningTime="2026-07-09 22:09:00.481755338 +0000 UTC m=+8.579970522" Jul 09 22:09:01.937002 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:01.936966 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:01.937386 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:01.937098 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:09:01.937386 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:01.937162 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:09.937144513 +0000 UTC m=+18.035359685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:09:02.038209 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:02.038174 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:02.038354 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:02.038314 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:09:02.038409 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:02.038378 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret podName:b276e595-42b4-452f-8c2b-703a2f894f07 nodeName:}" failed. No retries permitted until 2026-07-09 22:09:10.038360087 +0000 UTC m=+18.136575280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret") pod "global-pull-secret-syncer-vm5vl" (UID: "b276e595-42b4-452f-8c2b-703a2f894f07") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:09:02.138740 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:02.138688 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:02.138917 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:02.138906 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:09:02.139012 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:02.138925 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:09:02.139012 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:02.138938 2576 projected.go:196] Error preparing data for projected volume kube-api-access-fbkb5 for pod openshift-network-diagnostics/network-check-target-bqdc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:09:02.139012 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:02.138997 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5 podName:1a1bfaf6-12e3-4a89-b415-a5de9061cbba nodeName:}" failed. No retries permitted until 2026-07-09 22:09:10.138979014 +0000 UTC m=+18.237194217 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbkb5" (UniqueName: "kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5") pod "network-check-target-bqdc5" (UID: "1a1bfaf6-12e3-4a89-b415-a5de9061cbba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:09:02.383010 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:02.382932 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:02.383010 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:02.382964 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:02.383220 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:02.382929 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:02.383220 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:02.383052 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:02.383220 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:02.383135 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:02.383220 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:02.383197 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:04.382875 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:04.382796 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:04.382875 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:04.382852 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:04.383394 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:04.382985 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:04.383394 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:04.383008 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:04.383394 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:04.383081 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:04.383549 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:04.383424 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:06.382295 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:06.382254 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:06.382295 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:06.382285 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:06.382764 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:06.382258 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:06.382764 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:06.382383 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:06.382764 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:06.382492 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:06.382764 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:06.382582 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:08.383019 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:08.382922 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:08.383019 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:08.382977 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:08.383497 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:08.382922 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:08.383497 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:08.383040 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:08.383497 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:08.383145 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:08.383497 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:08.383235 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:09.996445 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:09.996355 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:09.996935 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:09.996509 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:09:09.996935 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:09.996581 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:25.996566629 +0000 UTC m=+34.094781816 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:09:10.097768 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:10.097730 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:10.097937 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:10.097886 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:09:10.097992 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:10.097958 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret podName:b276e595-42b4-452f-8c2b-703a2f894f07 nodeName:}" failed. No retries permitted until 2026-07-09 22:09:26.097942592 +0000 UTC m=+34.196157763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret") pod "global-pull-secret-syncer-vm5vl" (UID: "b276e595-42b4-452f-8c2b-703a2f894f07") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:09:10.198249 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:10.198211 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:10.198412 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:10.198349 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:09:10.198412 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:10.198365 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:09:10.198412 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:10.198374 2576 projected.go:196] Error preparing data for projected volume kube-api-access-fbkb5 for pod openshift-network-diagnostics/network-check-target-bqdc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:09:10.198582 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:10.198427 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5 podName:1a1bfaf6-12e3-4a89-b415-a5de9061cbba nodeName:}" failed. No retries permitted until 2026-07-09 22:09:26.19840988 +0000 UTC m=+34.296625071 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbkb5" (UniqueName: "kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5") pod "network-check-target-bqdc5" (UID: "1a1bfaf6-12e3-4a89-b415-a5de9061cbba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:09:10.382295 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:10.382216 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:10.382295 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:10.382253 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:10.382495 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:10.382220 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:10.382495 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:10.382346 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:10.382495 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:10.382446 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:10.382613 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:10.382528 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:12.383330 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:12.383296 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:12.383831 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:12.383388 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:12.383831 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:12.383427 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:12.383831 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:12.383390 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:12.383831 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:12.383522 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:12.383831 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:12.383620 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:14.383255 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.383044 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:14.383849 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.383050 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:14.383849 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:14.383292 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:14.383849 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:14.383384 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:14.383849 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.383123 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:14.383849 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:14.383498 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:14.486053 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.486026 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" event={"ID":"fea577f9-5cea-46b9-9591-c88c1549dea5","Type":"ContainerStarted","Data":"da24c6307e782fde4cec713245fd86c2853d668965b185abc60fa3b66d3a7275"} Jul 09 22:09:14.486151 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.486094 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" event={"ID":"fea577f9-5cea-46b9-9591-c88c1549dea5","Type":"ContainerStarted","Data":"5ceafc3a4fe210c4e55f09bb70a8efc09eb3023a95323ddbdf0ad962c558b54b"} Jul 09 22:09:14.486151 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.486138 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" event={"ID":"fea577f9-5cea-46b9-9591-c88c1549dea5","Type":"ContainerStarted","Data":"570fd8d6c1f46265ea58125e880909b879173cb1edf94015ab5ad4aeb55f9775"} Jul 09 22:09:14.486236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.486167 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" event={"ID":"fea577f9-5cea-46b9-9591-c88c1549dea5","Type":"ContainerStarted","Data":"0ad858f80c3204554f0746cec2147d569d7915a82474c78ed346dbc56c8c250a"} Jul 09 22:09:14.487303 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.487273 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lj48t" event={"ID":"7e53afda-97f8-4484-9c7c-b747bde49e8f","Type":"ContainerStarted","Data":"1f309bcf6e90c9a86274d3455fdc30abecbd7952cd2bccd4423bf35c179fdec6"} Jul 09 22:09:14.488659 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.488641 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w75wq" event={"ID":"4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5","Type":"ContainerStarted","Data":"4a69e9896d5b41f2ba8e75d4bb9788e3d5a901887e383adaee95fcf986ffe86b"} Jul 09 22:09:14.489998 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.489973 2576 generic.go:356] "Generic (PLEG): container finished" podID="976f8a42-342d-4902-8d9a-1c6753f67d83" containerID="a4484ec259b4379a5c5329c95fcdde3e370806411e5aa1408b928c99094cc814" exitCode=0 Jul 09 22:09:14.490077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.490057 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" event={"ID":"976f8a42-342d-4902-8d9a-1c6753f67d83","Type":"ContainerDied","Data":"a4484ec259b4379a5c5329c95fcdde3e370806411e5aa1408b928c99094cc814"} Jul 09 22:09:14.491405 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.491388 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" event={"ID":"1ba084a3-ee75-4a21-b72c-5f7fe4e74fd4","Type":"ContainerStarted","Data":"cf220cbc7ff3d7e708b5baf714441b882b994545fb1979e40523e261a04cece8"} Jul 09 22:09:14.493129 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.493111 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" event={"ID":"ce1d60138de5db8ac5067e9c282bb219","Type":"ContainerStarted","Data":"a6adb51f513052495aa8807691addeb45a1b29ed1bf38f7620352d9218a6feca"} Jul 09 22:09:14.494561 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.494543 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zxpng" event={"ID":"bd4e48cc-2d92-4a9c-a23f-e25fe939fd87","Type":"ContainerStarted","Data":"abb1bd8a6ca0609ab829fecdf917235ac3b4e2eabac3a17d81b4a69ca88c9c78"} Jul 09 22:09:14.495870 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.495853 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-65pnm" event={"ID":"dff8d8d0-93d0-40a2-8061-9fb8f8903316","Type":"ContainerStarted","Data":"9e64dbb0d4256dc9908b9457380067784bf4784223770b2549103b60a661b376"} Jul 09 22:09:14.497118 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.497094 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" event={"ID":"09ffcc38-f771-46ba-acab-e1c4a45eedda","Type":"ContainerStarted","Data":"70115c721297c76eeecaa0d77af86348ca7e886e6cc6b11ca778b8519dff38a6"} Jul 09 22:09:14.525289 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.525244 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lj48t" podStartSLOduration=4.083639534 podStartE2EDuration="21.525230076s" totalImagesPullingTime="17.441590542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:14.50493456 +0000 UTC m=+22.603149763" watchObservedRunningTime="2026-07-09 22:09:14.525230076 +0000 UTC m=+22.623445266" Jul 09 22:09:14.525563 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.525538 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-244.ec2.internal" podStartSLOduration=21.525530352 podStartE2EDuration="21.525530352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:14.525137917 +0000 UTC m=+22.623353107" watchObservedRunningTime="2026-07-09 22:09:14.525530352 +0000 UTC m=+22.623745542" Jul 09 22:09:14.589543 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.589503 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zxpng" podStartSLOduration=3.27547309 podStartE2EDuration="22.5894876s" totalImagesPullingTime="19.31401451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:14.58904426 +0000 UTC m=+22.687259450" watchObservedRunningTime="2026-07-09 22:09:14.5894876 +0000 UTC m=+22.687702789" Jul 09 22:09:14.589829 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.589807 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w75wq" podStartSLOduration=3.313361307 podStartE2EDuration="22.589800888s" totalImagesPullingTime="19.276439581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:14.569411405 +0000 UTC m=+22.667626594" watchObservedRunningTime="2026-07-09 22:09:14.589800888 +0000 UTC m=+22.688016098" Jul 09 22:09:14.610772 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.610704 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xd2gj" podStartSLOduration=3.307526288 podStartE2EDuration="22.610685135s" totalImagesPullingTime="19.303158847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:14.610226873 +0000 UTC m=+22.708442062" watchObservedRunningTime="2026-07-09 22:09:14.610685135 +0000 UTC m=+22.708900326" Jul 09 22:09:14.631372 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:14.631321 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-65pnm" podStartSLOduration=3.371543279 podStartE2EDuration="22.631303313s" totalImagesPullingTime="19.259760034s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:14.631265133 +0000 UTC m=+22.729480324" watchObservedRunningTime="2026-07-09 22:09:14.631303313 +0000 UTC m=+22.729518504" Jul 09 22:09:15.311448 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:15.311416 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:09:15.312172 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:15.312156 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:09:15.502498 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:15.502457 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j7999" event={"ID":"210ead62-927e-4f84-82b3-d26033a4219a","Type":"ContainerStarted","Data":"2e5baf44173aca4c77ecfffe552beb5da4f1ae44c50b65c8758a01c01e79ea47"} Jul 09 22:09:15.507121 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:15.507081 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" event={"ID":"fea577f9-5cea-46b9-9591-c88c1549dea5","Type":"ContainerStarted","Data":"d4b801fa0c6a738b3bd1d78a591b4901e6be76e56e9fc7d6bc73778b45529982"} Jul 09 22:09:15.507999 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:15.507978 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:09:15.508123 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:15.508027 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w75wq" Jul 09 22:09:15.508123 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:15.508046 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" event={"ID":"fea577f9-5cea-46b9-9591-c88c1549dea5","Type":"ContainerStarted","Data":"a8232312c86242d73e8632cca46d9d977ca3503f838737899f1cd29b6d6de417"} Jul 09 22:09:15.521837 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:15.521801 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j7999" podStartSLOduration=3.319425191 podStartE2EDuration="22.52178792s" totalImagesPullingTime="19.202362729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:15.521536394 +0000 UTC m=+23.619751585" watchObservedRunningTime="2026-07-09 22:09:15.52178792 +0000 UTC m=+23.620003109" Jul 09 22:09:15.574486 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:15.574460 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 22:09:16.337473 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:16.337336 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T22:09:15.574475995Z","UUID":"ff2cc974-9461-43ee-90c9-66e65224f2cd","Handler":null,"Name":"","Endpoint":""} Jul 09 22:09:16.338733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:16.338697 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 22:09:16.338858 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:16.338752 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 22:09:16.383003 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:16.382971 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:16.383173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:16.382983 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:16.383173 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:16.383099 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:16.383173 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:16.383163 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:16.383285 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:16.383206 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:16.383285 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:16.383258 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:16.511181 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:16.511147 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" event={"ID":"09ffcc38-f771-46ba-acab-e1c4a45eedda","Type":"ContainerStarted","Data":"ecc98b1570527163ca65709b8de8c117d059e4e90e12dfc86f711e6b51d95b1c"} Jul 09 22:09:17.515936 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:17.515848 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" event={"ID":"09ffcc38-f771-46ba-acab-e1c4a45eedda","Type":"ContainerStarted","Data":"7369c093bb470ccf801e708b70549fffcfb13dc4798b7a3cf94486221605ba89"} Jul 09 22:09:17.519677 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:17.519646 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" event={"ID":"fea577f9-5cea-46b9-9591-c88c1549dea5","Type":"ContainerStarted","Data":"cc3a03e50d37f2fac6cc5f4e89b82fbc104df494d731755cad8be94e49687154"} Jul 09 22:09:17.542598 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:17.542402 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jlkmx" podStartSLOduration=3.521290557 podStartE2EDuration="25.542381219s" totalImagesPullingTime="22.021090662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:17.542288152 +0000 UTC m=+25.640503343" watchObservedRunningTime="2026-07-09 22:09:17.542381219 +0000 UTC m=+25.640596410" Jul 09 22:09:18.382384 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:18.382351 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:18.382572 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:18.382402 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:18.382572 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:18.382351 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:18.382572 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:18.382490 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:18.382752 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:18.382574 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:18.382752 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:18.382644 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:20.382451 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.382418 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:20.382451 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.382445 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:20.383296 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:20.382538 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:20.383296 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:20.382580 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:20.383296 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.382599 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:20.383296 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:20.382673 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:20.530090 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.529963 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" event={"ID":"fea577f9-5cea-46b9-9591-c88c1549dea5","Type":"ContainerStarted","Data":"a04c1e6812adc832e8d030f1fb958d1379baa7075e7ec97b42d020a132704717"} Jul 09 22:09:20.530480 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.530264 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:09:20.530480 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.530329 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:09:20.530480 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.530380 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:09:20.546950 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.546928 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:09:20.547125 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.547111 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:09:20.575402 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:20.575357 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" podStartSLOduration=9.190820504 podStartE2EDuration="28.575344057s" totalImagesPullingTime="19.384523553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:20.574880215 +0000 UTC m=+28.673095409" watchObservedRunningTime="2026-07-09 22:09:20.575344057 +0000 UTC m=+28.673559247" Jul 09 22:09:21.533829 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:21.533632 2576 generic.go:356] "Generic (PLEG): container finished" podID="976f8a42-342d-4902-8d9a-1c6753f67d83" containerID="acf24f7e7dfe816e245363b8720e87f232929bd819d50d50fb2473fdeae5659f" exitCode=0 Jul 09 22:09:21.534517 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:21.533711 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" event={"ID":"976f8a42-342d-4902-8d9a-1c6753f67d83","Type":"ContainerDied","Data":"acf24f7e7dfe816e245363b8720e87f232929bd819d50d50fb2473fdeae5659f"} Jul 09 22:09:22.383694 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:22.383663 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:22.383912 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:22.383774 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:22.383912 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:22.383804 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:22.383912 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:22.383895 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:22.384073 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:22.383947 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:22.384073 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:22.384013 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:22.643280 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:22.643124 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bqdc5"] Jul 09 22:09:22.643672 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:22.643345 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:22.643672 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:22.643426 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:22.648619 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:22.648597 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6969j"] Jul 09 22:09:22.648737 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:22.648684 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:22.648803 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:22.648787 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:22.667156 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:22.667134 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vm5vl"] Jul 09 22:09:22.667258 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:22.667217 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:22.667306 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:22.667291 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:23.540413 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:23.540380 2576 generic.go:356] "Generic (PLEG): container finished" podID="976f8a42-342d-4902-8d9a-1c6753f67d83" containerID="1d9bc82c4c7eedda2653544a0f717b84b5935ec72af0f111f86f825f1315946e" exitCode=0 Jul 09 22:09:23.540570 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:23.540422 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" event={"ID":"976f8a42-342d-4902-8d9a-1c6753f67d83","Type":"ContainerDied","Data":"1d9bc82c4c7eedda2653544a0f717b84b5935ec72af0f111f86f825f1315946e"} Jul 09 22:09:24.382450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:24.382424 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:24.382450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:24.382441 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:24.382980 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:24.382461 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:24.382980 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:24.382525 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:24.382980 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:24.382593 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:24.382980 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:24.382633 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:25.547866 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:25.547777 2576 generic.go:356] "Generic (PLEG): container finished" podID="976f8a42-342d-4902-8d9a-1c6753f67d83" containerID="0491db27569026bf77439810c7d8ba352ebe77d8c0b677041544e93785b763b3" exitCode=0 Jul 09 22:09:25.547866 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:25.547837 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" event={"ID":"976f8a42-342d-4902-8d9a-1c6753f67d83","Type":"ContainerDied","Data":"0491db27569026bf77439810c7d8ba352ebe77d8c0b677041544e93785b763b3"} Jul 09 22:09:26.010576 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.010499 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:26.010711 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.010594 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:09:26.010711 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.010642 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:58.010628538 +0000 UTC m=+66.108843707 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 22:09:26.111621 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.111589 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:26.111808 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.111777 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 22:09:26.111884 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.111859 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret podName:b276e595-42b4-452f-8c2b-703a2f894f07 nodeName:}" failed. No retries permitted until 2026-07-09 22:09:58.111837975 +0000 UTC m=+66.210053163 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret") pod "global-pull-secret-syncer-vm5vl" (UID: "b276e595-42b4-452f-8c2b-703a2f894f07") : object "kube-system"/"original-pull-secret" not registered Jul 09 22:09:26.212730 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.212680 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:26.212891 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.212842 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 22:09:26.212891 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.212863 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 22:09:26.212891 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.212876 2576 projected.go:196] Error preparing data for projected volume kube-api-access-fbkb5 for pod openshift-network-diagnostics/network-check-target-bqdc5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:09:26.213079 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.212952 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5 podName:1a1bfaf6-12e3-4a89-b415-a5de9061cbba nodeName:}" failed. No retries permitted until 2026-07-09 22:09:58.212938294 +0000 UTC m=+66.311153462 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbkb5" (UniqueName: "kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5") pod "network-check-target-bqdc5" (UID: "1a1bfaf6-12e3-4a89-b415-a5de9061cbba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 22:09:26.382373 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.382278 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:26.382373 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.382278 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:26.382586 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.382397 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm5vl" podUID="b276e595-42b4-452f-8c2b-703a2f894f07" Jul 09 22:09:26.382586 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.382278 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:26.382586 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.382459 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:09:26.382586 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.382533 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bqdc5" podUID="1a1bfaf6-12e3-4a89-b415-a5de9061cbba" Jul 09 22:09:26.714619 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.714544 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-244.ec2.internal" event="NodeReady" Jul 09 22:09:26.715031 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.714698 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 22:09:26.851617 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.851581 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dcgsk"] Jul 09 22:09:26.855754 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.855732 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:26.857827 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.857798 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rxhvj"] Jul 09 22:09:26.862265 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.861877 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rxhvj" Jul 09 22:09:26.862265 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.861915 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-dockercfg-77st2\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-dns\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77st2\"" type="*v1.Secret" Jul 09 22:09:26.862265 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.862012 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-default-metrics-tls\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-dns\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret" Jul 09 22:09:26.862265 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.862131 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"dns-default\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-dns\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap" Jul 09 22:09:26.862483 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:26.862356 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"dns-default-dcgsk\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-dns\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c" pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:26.872707 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.872686 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 22:09:26.873030 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.873013 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rkrl7\"" Jul 09 22:09:26.874237 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.874214 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 22:09:26.874331 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.874319 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 22:09:26.893465 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.893406 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dcgsk"] Jul 09 22:09:26.893693 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:26.893678 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rxhvj"] Jul 09 22:09:27.017961 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.017867 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv5kr\" (UniqueName: \"kubernetes.io/projected/e252f3c5-1f99-4dd1-8d97-8d666602d58c-kube-api-access-fv5kr\") pod \"ingress-canary-rxhvj\" (UID: \"e252f3c5-1f99-4dd1-8d97-8d666602d58c\") " pod="openshift-ingress-canary/ingress-canary-rxhvj" Jul 09 22:09:27.017961 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.017944 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-config-volume\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.018171 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.017975 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-tmp-dir\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.018171 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.018003 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gk7\" (UniqueName: \"kubernetes.io/projected/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-kube-api-access-p2gk7\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.018171 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.018028 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.018171 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.018088 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e252f3c5-1f99-4dd1-8d97-8d666602d58c-cert\") pod \"ingress-canary-rxhvj\" (UID: \"e252f3c5-1f99-4dd1-8d97-8d666602d58c\") " pod="openshift-ingress-canary/ingress-canary-rxhvj" Jul 09 22:09:27.119393 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.119352 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv5kr\" (UniqueName: \"kubernetes.io/projected/e252f3c5-1f99-4dd1-8d97-8d666602d58c-kube-api-access-fv5kr\") pod \"ingress-canary-rxhvj\" (UID: \"e252f3c5-1f99-4dd1-8d97-8d666602d58c\") " pod="openshift-ingress-canary/ingress-canary-rxhvj" Jul 09 22:09:27.119561 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.119409 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-config-volume\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.119561 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.119436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-tmp-dir\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.119561 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.119458 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gk7\" (UniqueName: \"kubernetes.io/projected/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-kube-api-access-p2gk7\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.119561 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.119476 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.119561 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.119509 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e252f3c5-1f99-4dd1-8d97-8d666602d58c-cert\") pod \"ingress-canary-rxhvj\" (UID: \"e252f3c5-1f99-4dd1-8d97-8d666602d58c\") " pod="openshift-ingress-canary/ingress-canary-rxhvj" Jul 09 22:09:27.119892 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.119861 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-tmp-dir\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.123898 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.123875 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e252f3c5-1f99-4dd1-8d97-8d666602d58c-cert\") pod \"ingress-canary-rxhvj\" (UID: \"e252f3c5-1f99-4dd1-8d97-8d666602d58c\") " pod="openshift-ingress-canary/ingress-canary-rxhvj" Jul 09 22:09:27.132104 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.132082 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gk7\" (UniqueName: \"kubernetes.io/projected/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-kube-api-access-p2gk7\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:27.132569 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.132549 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv5kr\" (UniqueName: \"kubernetes.io/projected/e252f3c5-1f99-4dd1-8d97-8d666602d58c-kube-api-access-fv5kr\") pod \"ingress-canary-rxhvj\" (UID: \"e252f3c5-1f99-4dd1-8d97-8d666602d58c\") " pod="openshift-ingress-canary/ingress-canary-rxhvj" Jul 09 22:09:27.173059 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.173023 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rxhvj" Jul 09 22:09:27.322321 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.322067 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rxhvj"] Jul 09 22:09:27.325514 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:09:27.325486 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode252f3c5_1f99_4dd1_8d97_8d666602d58c.slice/crio-5a1240ebe983d8214748980a7c25b8904f7b41ca05b2b7b97d6f3bedb7b81fa8 WatchSource:0}: Error finding container 5a1240ebe983d8214748980a7c25b8904f7b41ca05b2b7b97d6f3bedb7b81fa8: Status 404 returned error can't find the container with id 5a1240ebe983d8214748980a7c25b8904f7b41ca05b2b7b97d6f3bedb7b81fa8 Jul 09 22:09:27.553215 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.553183 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rxhvj" event={"ID":"e252f3c5-1f99-4dd1-8d97-8d666602d58c","Type":"ContainerStarted","Data":"5a1240ebe983d8214748980a7c25b8904f7b41ca05b2b7b97d6f3bedb7b81fa8"} Jul 09 22:09:27.906243 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:27.906208 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 22:09:27.910490 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:27.910467 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:09:27.910610 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:27.910529 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls podName:6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:28.410509774 +0000 UTC m=+36.508724941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls") pod "dns-default-dcgsk" (UID: "6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c") : secret "dns-default-metrics-tls" not found Jul 09 22:09:28.117710 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.117627 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 22:09:28.120859 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.120829 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-config-volume\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:28.306866 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.306830 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77st2\"" Jul 09 22:09:28.382846 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.382771 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:28.383006 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.382771 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:28.383372 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.382771 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:28.386831 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.386804 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 22:09:28.387116 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.387098 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zm9fl\"" Jul 09 22:09:28.388768 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.388748 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k7bfs\"" Jul 09 22:09:28.388852 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.388772 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 22:09:28.388852 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.388755 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 22:09:28.388972 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.388851 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 22:09:28.429767 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:28.429740 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:28.429952 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:28.429910 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:09:28.430005 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:28.429981 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls podName:6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:29.429961648 +0000 UTC m=+37.528176822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls") pod "dns-default-dcgsk" (UID: "6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c") : secret "dns-default-metrics-tls" not found Jul 09 22:09:29.438148 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:29.438114 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:29.438548 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:29.438281 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:09:29.438548 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:29.438366 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls podName:6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:31.438343858 +0000 UTC m=+39.536559058 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls") pod "dns-default-dcgsk" (UID: "6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c") : secret "dns-default-metrics-tls" not found Jul 09 22:09:31.454407 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:31.454371 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:31.454840 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:31.454480 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:09:31.454840 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:31.454531 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls podName:6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:35.454518265 +0000 UTC m=+43.552733432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls") pod "dns-default-dcgsk" (UID: "6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c") : secret "dns-default-metrics-tls" not found Jul 09 22:09:32.567146 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:32.567062 2576 generic.go:356] "Generic (PLEG): container finished" podID="976f8a42-342d-4902-8d9a-1c6753f67d83" containerID="458cebe8cea6c01dd2c7d7e8359c3106446516554f757717409e61b5d214798f" exitCode=0 Jul 09 22:09:32.567558 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:32.567141 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" event={"ID":"976f8a42-342d-4902-8d9a-1c6753f67d83","Type":"ContainerDied","Data":"458cebe8cea6c01dd2c7d7e8359c3106446516554f757717409e61b5d214798f"} Jul 09 22:09:32.568621 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:32.568590 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rxhvj" event={"ID":"e252f3c5-1f99-4dd1-8d97-8d666602d58c","Type":"ContainerStarted","Data":"2f4e683901417dbd9d22278ade9cc223e0a6363f8c229c9d2b508efc198f4665"} Jul 09 22:09:32.629077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:32.629021 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rxhvj" podStartSLOduration=2.315858233 podStartE2EDuration="6.628999735s" totalImagesPullingTime="4.313141502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:32.627845065 +0000 UTC m=+40.726060255" watchObservedRunningTime="2026-07-09 22:09:32.628999735 +0000 UTC m=+40.727215132" Jul 09 22:09:33.573842 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:33.573809 2576 generic.go:356] "Generic (PLEG): container finished" podID="976f8a42-342d-4902-8d9a-1c6753f67d83" containerID="5d6e752bc88538b09acced3c9a32d4ffcf6432d149f09b697ae2c3e3a8be4405" exitCode=0 Jul 09 22:09:33.574330 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:33.573895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" event={"ID":"976f8a42-342d-4902-8d9a-1c6753f67d83","Type":"ContainerDied","Data":"5d6e752bc88538b09acced3c9a32d4ffcf6432d149f09b697ae2c3e3a8be4405"} Jul 09 22:09:34.579546 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:34.579513 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" event={"ID":"976f8a42-342d-4902-8d9a-1c6753f67d83","Type":"ContainerStarted","Data":"648266f918d155c20553f1a9c0502a7336f160708ae31595fbd8ec0bdd14d0f2"} Jul 09 22:09:34.620873 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:34.620822 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5jqh2" podStartSLOduration=8.854538737 podStartE2EDuration="42.62080666s" totalImagesPullingTime="33.766267923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:34.620394965 +0000 UTC m=+42.718610155" watchObservedRunningTime="2026-07-09 22:09:34.62080666 +0000 UTC m=+42.719021874" Jul 09 22:09:35.481727 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:35.481687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:35.481914 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:35.481830 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:09:35.481914 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:35.481889 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls podName:6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:43.48187408 +0000 UTC m=+51.580089247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls") pod "dns-default-dcgsk" (UID: "6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c") : secret "dns-default-metrics-tls" not found Jul 09 22:09:43.535262 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:43.535225 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:43.535741 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:43.535334 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:09:43.535741 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:43.535382 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls podName:6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c nodeName:}" failed. No retries permitted until 2026-07-09 22:09:59.535369441 +0000 UTC m=+67.633584608 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls") pod "dns-default-dcgsk" (UID: "6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c") : secret "dns-default-metrics-tls" not found Jul 09 22:09:52.554362 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:52.554334 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7clnl" Jul 09 22:09:58.024346 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.024298 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:09:58.028209 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.028190 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 22:09:58.035069 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:58.035051 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 22:09:58.035131 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:58.035120 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:11:02.035102102 +0000 UTC m=+130.133317283 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : secret "metrics-daemon-secret" not found Jul 09 22:09:58.124888 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.124854 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:58.128575 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.128557 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 22:09:58.138908 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.138889 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b276e595-42b4-452f-8c2b-703a2f894f07-original-pull-secret\") pod \"global-pull-secret-syncer-vm5vl\" (UID: \"b276e595-42b4-452f-8c2b-703a2f894f07\") " pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:58.225317 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.225284 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:58.229078 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.229060 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 22:09:58.239507 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.239491 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 22:09:58.248901 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.248885 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbkb5\" (UniqueName: \"kubernetes.io/projected/1a1bfaf6-12e3-4a89-b415-a5de9061cbba-kube-api-access-fbkb5\") pod \"network-check-target-bqdc5\" (UID: \"1a1bfaf6-12e3-4a89-b415-a5de9061cbba\") " pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:58.399653 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.399623 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k7bfs\"" Jul 09 22:09:58.406863 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.406842 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:09:58.408080 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.408065 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm5vl" Jul 09 22:09:58.564661 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.564620 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vm5vl"] Jul 09 22:09:58.566848 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:09:58.566812 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb276e595_42b4_452f_8c2b_703a2f894f07.slice/crio-9834e5e576f6970a09f5b2d0daf369e06564a397f8f5f410e85c617b7ad5c3f5 WatchSource:0}: Error finding container 9834e5e576f6970a09f5b2d0daf369e06564a397f8f5f410e85c617b7ad5c3f5: Status 404 returned error can't find the container with id 9834e5e576f6970a09f5b2d0daf369e06564a397f8f5f410e85c617b7ad5c3f5 Jul 09 22:09:58.568426 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.568326 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bqdc5"] Jul 09 22:09:58.570711 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:09:58.570689 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a1bfaf6_12e3_4a89_b415_a5de9061cbba.slice/crio-e608aed92062ba350afc09a9e7b285b5dad9d99a6185f4f56c234986d2691858 WatchSource:0}: Error finding container e608aed92062ba350afc09a9e7b285b5dad9d99a6185f4f56c234986d2691858: Status 404 returned error can't find the container with id e608aed92062ba350afc09a9e7b285b5dad9d99a6185f4f56c234986d2691858 Jul 09 22:09:58.634906 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.634857 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vm5vl" event={"ID":"b276e595-42b4-452f-8c2b-703a2f894f07","Type":"ContainerStarted","Data":"9834e5e576f6970a09f5b2d0daf369e06564a397f8f5f410e85c617b7ad5c3f5"} Jul 09 22:09:58.635918 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:58.635899 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bqdc5" event={"ID":"1a1bfaf6-12e3-4a89-b415-a5de9061cbba","Type":"ContainerStarted","Data":"e608aed92062ba350afc09a9e7b285b5dad9d99a6185f4f56c234986d2691858"} Jul 09 22:09:59.635432 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:09:59.635396 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:09:59.636306 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:59.635534 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:09:59.636306 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:09:59.635601 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls podName:6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c nodeName:}" failed. No retries permitted until 2026-07-09 22:10:31.635582115 +0000 UTC m=+99.733797289 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls") pod "dns-default-dcgsk" (UID: "6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c") : secret "dns-default-metrics-tls" not found Jul 09 22:10:04.653002 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:10:04.652964 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vm5vl" event={"ID":"b276e595-42b4-452f-8c2b-703a2f894f07","Type":"ContainerStarted","Data":"f4bf7219bf2354929283119266ee88980a946e60eb52034a0b3281ebf42a56b8"} Jul 09 22:10:04.654291 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:10:04.654266 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bqdc5" event={"ID":"1a1bfaf6-12e3-4a89-b415-a5de9061cbba","Type":"ContainerStarted","Data":"b6a9326b3383bca10a8bf8328c074f5982c1a8f99042319f55b23dfeb8acc5c1"} Jul 09 22:10:04.654419 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:10:04.654398 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:10:04.686183 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:10:04.686140 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vm5vl" podStartSLOduration=66.474725699 podStartE2EDuration="1m11.686126458s" totalImagesPullingTime="5.211400759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:04.685230912 +0000 UTC m=+72.783446102" watchObservedRunningTime="2026-07-09 22:10:04.686126458 +0000 UTC m=+72.784341648" Jul 09 22:10:04.716484 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:10:04.716439 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bqdc5" podStartSLOduration=66.513516149 podStartE2EDuration="1m11.716425804s" totalImagesPullingTime="5.202909655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:10:04.716295618 +0000 UTC m=+72.814510808" watchObservedRunningTime="2026-07-09 22:10:04.716425804 +0000 UTC m=+72.814641037" Jul 09 22:10:31.654279 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:10:31.654239 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:10:31.654820 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:10:31.654345 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:10:31.654820 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:10:31.654408 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls podName:6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c nodeName:}" failed. No retries permitted until 2026-07-09 22:11:35.654394486 +0000 UTC m=+163.752609654 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls") pod "dns-default-dcgsk" (UID: "6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c") : secret "dns-default-metrics-tls" not found Jul 09 22:10:35.658827 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:10:35.658797 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bqdc5" Jul 09 22:11:00.846966 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.846920 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk"] Jul 09 22:11:00.849749 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.849708 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" Jul 09 22:11:00.855671 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.855297 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 22:11:00.855671 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.855374 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 22:11:00.855837 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.855713 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-lt4wd\"" Jul 09 22:11:00.855837 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.855781 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 22:11:00.855837 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.855835 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 22:11:00.859408 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.859383 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk"] Jul 09 22:11:00.951060 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.951021 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qwx6\" (UniqueName: \"kubernetes.io/projected/c8dc1300-7d4e-4aef-89ec-74fd5e7c9519-kube-api-access-4qwx6\") pod \"managed-serviceaccount-addon-agent-85fc4456fb-995kk\" (UID: \"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" Jul 09 22:11:00.951241 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:00.951077 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c8dc1300-7d4e-4aef-89ec-74fd5e7c9519-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85fc4456fb-995kk\" (UID: \"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" Jul 09 22:11:01.051882 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:01.051843 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qwx6\" (UniqueName: \"kubernetes.io/projected/c8dc1300-7d4e-4aef-89ec-74fd5e7c9519-kube-api-access-4qwx6\") pod \"managed-serviceaccount-addon-agent-85fc4456fb-995kk\" (UID: \"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" Jul 09 22:11:01.052030 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:01.051893 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c8dc1300-7d4e-4aef-89ec-74fd5e7c9519-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85fc4456fb-995kk\" (UID: \"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" Jul 09 22:11:01.054373 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:01.054349 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c8dc1300-7d4e-4aef-89ec-74fd5e7c9519-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85fc4456fb-995kk\" (UID: \"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" Jul 09 22:11:01.063860 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:01.063830 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qwx6\" (UniqueName: \"kubernetes.io/projected/c8dc1300-7d4e-4aef-89ec-74fd5e7c9519-kube-api-access-4qwx6\") pod \"managed-serviceaccount-addon-agent-85fc4456fb-995kk\" (UID: \"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" Jul 09 22:11:01.171650 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:01.171555 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" Jul 09 22:11:01.288761 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:01.288674 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk"] Jul 09 22:11:01.290633 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:11:01.290609 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8dc1300_7d4e_4aef_89ec_74fd5e7c9519.slice/crio-513dfe727e5830f1f5423423a5292bb6801b89ed24c0c291e1e450b497034ecb WatchSource:0}: Error finding container 513dfe727e5830f1f5423423a5292bb6801b89ed24c0c291e1e450b497034ecb: Status 404 returned error can't find the container with id 513dfe727e5830f1f5423423a5292bb6801b89ed24c0c291e1e450b497034ecb Jul 09 22:11:01.797195 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:01.797154 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" event={"ID":"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519","Type":"ContainerStarted","Data":"513dfe727e5830f1f5423423a5292bb6801b89ed24c0c291e1e450b497034ecb"} Jul 09 22:11:02.059089 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:02.058973 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:11:02.059528 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:11:02.059155 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 22:11:02.059528 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:11:02.059228 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:13:04.059205508 +0000 UTC m=+252.157420691 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : secret "metrics-daemon-secret" not found Jul 09 22:11:03.803973 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:03.803882 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" event={"ID":"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519","Type":"ContainerStarted","Data":"7fe1a5e07296e20b7eba4192bcfde65348b731f898de1c9de79c68b11ae13021"} Jul 09 22:11:03.821016 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:03.820970 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" podStartSLOduration=1.664263424 podStartE2EDuration="3.820954439s" totalImagesPullingTime="2.156691015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:11:03.820433394 +0000 UTC m=+131.918648583" watchObservedRunningTime="2026-07-09 22:11:03.820954439 +0000 UTC m=+131.919169629" Jul 09 22:11:29.867075 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:11:29.867039 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dcgsk" podUID="6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c" Jul 09 22:11:30.866478 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:30.866449 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dcgsk" Jul 09 22:11:31.402115 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:11:31.402081 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:11:35.693962 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:35.693909 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:11:35.694569 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:11:35.694077 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 22:11:35.694569 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:11:35.694158 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls podName:6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c nodeName:}" failed. No retries permitted until 2026-07-09 22:13:37.694137019 +0000 UTC m=+285.792352205 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls") pod "dns-default-dcgsk" (UID: "6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c") : secret "dns-default-metrics-tls" not found Jul 09 22:11:46.383232 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:11:46.383146 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:12:55.977345 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:55.977315 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-58c6fd45c7-x2b2x"] Jul 09 22:12:55.980128 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:55.980113 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:55.986617 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:55.986590 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 22:12:55.987656 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:55.987632 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 09 22:12:55.987758 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:55.987644 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 09 22:12:55.987906 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:55.987895 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 09 22:12:55.988575 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:55.988563 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 22:12:55.988614 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:55.988587 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qtrdb\"" Jul 09 22:12:55.988651 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:55.988587 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 09 22:12:56.015325 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.015294 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58c6fd45c7-x2b2x"] Jul 09 22:12:56.028782 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.028755 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-default-certificate\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.028894 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.028789 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.028894 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.028809 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.028894 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.028855 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfdzn\" (UniqueName: \"kubernetes.io/projected/e28edf89-6b16-4538-97f4-7f72e368a48d-kube-api-access-hfdzn\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.028995 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.028924 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-stats-auth\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.055231 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.055203 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv"] Jul 09 22:12:56.057967 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.057952 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:12:56.062313 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.062291 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 09 22:12:56.062425 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.062349 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 09 22:12:56.062425 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.062375 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bsjcp\"" Jul 09 22:12:56.064611 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.064593 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nr6vw"] Jul 09 22:12:56.065178 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.065164 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 09 22:12:56.067213 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.067194 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.070235 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.070217 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56d94fb54-sbs7x"] Jul 09 22:12:56.071358 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.071336 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 22:12:56.071358 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.071352 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 22:12:56.071852 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.071836 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 22:12:56.071958 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.071889 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 22:12:56.071958 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.071953 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kwqj5\"" Jul 09 22:12:56.072872 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.072853 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.077928 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.077891 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 22:12:56.082258 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.082238 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 22:12:56.082856 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.082839 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jg4kh\"" Jul 09 22:12:56.083441 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.083423 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 22:12:56.083441 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.083438 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 22:12:56.084022 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.084000 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv"] Jul 09 22:12:56.087957 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.087939 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nr6vw"] Jul 09 22:12:56.092840 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.092819 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 22:12:56.099395 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.099373 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d94fb54-sbs7x"] Jul 09 22:12:56.129455 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129426 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xd8s\" (UniqueName: \"kubernetes.io/projected/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-kube-api-access-6xd8s\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.129611 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129468 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-trusted-ca\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.129611 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129498 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:12:56.129611 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129545 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-default-certificate\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.129611 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129569 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.129611 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129593 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129617 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-serving-cert\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.129683 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.129685 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:12:56.629670166 +0000 UTC m=+244.727885353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : configmap references non-existent config key: service-ca.crt Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.129758 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:12:56.629742904 +0000 UTC m=+244.727958075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : secret "router-metrics-certs-default" not found Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129775 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfdzn\" (UniqueName: \"kubernetes.io/projected/e28edf89-6b16-4538-97f4-7f72e368a48d-kube-api-access-hfdzn\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129810 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-bound-sa-token\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-config\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129850 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-stats-auth\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.129896 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-installation-pull-secrets\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.130278 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129904 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d449e26d-5fdc-4014-864f-f1d1e89908f9-ca-trust-extracted\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.130278 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129931 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-trusted-ca\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.130278 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129969 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkbhs\" (UniqueName: \"kubernetes.io/projected/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-kube-api-access-bkbhs\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:12:56.130278 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.129988 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-certificates\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.130278 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.130007 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qcr6\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-kube-api-access-6qcr6\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.130278 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.130038 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-image-registry-private-configuration\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.132046 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.132022 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-default-certificate\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.132138 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.132045 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-stats-auth\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.152436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.152410 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfdzn\" (UniqueName: \"kubernetes.io/projected/e28edf89-6b16-4538-97f4-7f72e368a48d-kube-api-access-hfdzn\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.231274 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231201 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xd8s\" (UniqueName: \"kubernetes.io/projected/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-kube-api-access-6xd8s\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.231274 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-trusted-ca\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.231450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231279 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:12:56.231450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231329 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-serving-cert\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.231450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.231450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231374 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-bound-sa-token\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.231450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231400 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-config\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.231450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231429 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-installation-pull-secrets\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.231450 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.231445 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.231479 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.231500 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d94fb54-sbs7x: secret "image-registry-tls" not found Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.231519 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls podName:595f6c4e-df15-4da3-8fdf-a0cbc9cd6366 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:56.731498925 +0000 UTC m=+244.829714093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s5qhv" (UID: "595f6c4e-df15-4da3-8fdf-a0cbc9cd6366") : secret "samples-operator-tls" not found Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.231560 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls podName:d449e26d-5fdc-4014-864f-f1d1e89908f9 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:56.731543669 +0000 UTC m=+244.829758856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls") pod "image-registry-56d94fb54-sbs7x" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9") : secret "image-registry-tls" not found Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231592 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d449e26d-5fdc-4014-864f-f1d1e89908f9-ca-trust-extracted\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231626 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-trusted-ca\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231659 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbhs\" (UniqueName: \"kubernetes.io/projected/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-kube-api-access-bkbhs\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231693 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-certificates\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231735 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qcr6\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-kube-api-access-6qcr6\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.231821 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.231766 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-image-registry-private-configuration\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.232360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.232006 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d449e26d-5fdc-4014-864f-f1d1e89908f9-ca-trust-extracted\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.232360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.232188 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-config\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.232360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.232200 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-trusted-ca\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.232481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.232389 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-certificates\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.232545 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.232527 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-trusted-ca\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.233885 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.233867 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-installation-pull-secrets\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.234128 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.234109 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-image-registry-private-configuration\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.234430 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.234412 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-serving-cert\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.247458 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.247437 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qcr6\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-kube-api-access-6qcr6\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.247539 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.247495 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-bound-sa-token\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.249830 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.249803 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkbhs\" (UniqueName: \"kubernetes.io/projected/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-kube-api-access-bkbhs\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:12:56.250493 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.250474 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xd8s\" (UniqueName: \"kubernetes.io/projected/e8d556ba-1556-4990-a7fa-b6bd1399ed5b-kube-api-access-6xd8s\") pod \"console-operator-68b757fb67-nr6vw\" (UID: \"e8d556ba-1556-4990-a7fa-b6bd1399ed5b\") " pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.376826 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.376796 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:12:56.499055 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.499001 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-nr6vw"] Jul 09 22:12:56.501261 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:12:56.501224 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d556ba_1556_4990_a7fa_b6bd1399ed5b.slice/crio-5842c9ef0e8bc4faa8a089306eb5349004a62f0340fbbb07e45e3bc96560d7d3 WatchSource:0}: Error finding container 5842c9ef0e8bc4faa8a089306eb5349004a62f0340fbbb07e45e3bc96560d7d3: Status 404 returned error can't find the container with id 5842c9ef0e8bc4faa8a089306eb5349004a62f0340fbbb07e45e3bc96560d7d3 Jul 09 22:12:56.634375 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.634348 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.634530 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.634495 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 22:12:56.634572 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.634532 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:56.634572 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.634555 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:12:57.634539711 +0000 UTC m=+245.732754879 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : secret "router-metrics-certs-default" not found Jul 09 22:12:56.634646 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.634612 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:12:57.634598333 +0000 UTC m=+245.732813522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : configmap references non-existent config key: service-ca.crt Jul 09 22:12:56.735620 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.735585 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:12:56.735779 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:56.735639 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:56.735779 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.735729 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 22:12:56.735779 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.735770 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:12:56.735887 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.735781 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d94fb54-sbs7x: secret "image-registry-tls" not found Jul 09 22:12:56.735887 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.735786 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls podName:595f6c4e-df15-4da3-8fdf-a0cbc9cd6366 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:57.735772748 +0000 UTC m=+245.833987916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s5qhv" (UID: "595f6c4e-df15-4da3-8fdf-a0cbc9cd6366") : secret "samples-operator-tls" not found Jul 09 22:12:56.735887 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:56.735814 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls podName:d449e26d-5fdc-4014-864f-f1d1e89908f9 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:57.735804075 +0000 UTC m=+245.834019243 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls") pod "image-registry-56d94fb54-sbs7x" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9") : secret "image-registry-tls" not found Jul 09 22:12:57.070728 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:57.070684 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" event={"ID":"e8d556ba-1556-4990-a7fa-b6bd1399ed5b","Type":"ContainerStarted","Data":"5842c9ef0e8bc4faa8a089306eb5349004a62f0340fbbb07e45e3bc96560d7d3"} Jul 09 22:12:57.642657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:57.642621 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:57.642657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:57.642659 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:57.642872 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:57.642785 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 22:12:57.642872 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:57.642811 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:12:59.642792395 +0000 UTC m=+247.741007583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : configmap references non-existent config key: service-ca.crt Jul 09 22:12:57.642872 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:57.642833 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:12:59.642822883 +0000 UTC m=+247.741038057 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : secret "router-metrics-certs-default" not found Jul 09 22:12:57.743434 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:57.743393 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:12:57.743614 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:57.743565 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 22:12:57.743678 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:57.743629 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls podName:595f6c4e-df15-4da3-8fdf-a0cbc9cd6366 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:59.743614568 +0000 UTC m=+247.841829736 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s5qhv" (UID: "595f6c4e-df15-4da3-8fdf-a0cbc9cd6366") : secret "samples-operator-tls" not found Jul 09 22:12:57.743678 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:57.743648 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:57.744050 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:57.743823 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:12:57.744050 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:57.743837 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d94fb54-sbs7x: secret "image-registry-tls" not found Jul 09 22:12:57.744050 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:57.743883 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls podName:d449e26d-5fdc-4014-864f-f1d1e89908f9 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:59.74386983 +0000 UTC m=+247.842085011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls") pod "image-registry-56d94fb54-sbs7x" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9") : secret "image-registry-tls" not found Jul 09 22:12:59.659001 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:59.658965 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:59.659001 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:59.659006 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:12:59.659378 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:59.659103 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 22:12:59.659378 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:59.659162 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:13:03.659148135 +0000 UTC m=+251.757363315 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : secret "router-metrics-certs-default" not found Jul 09 22:12:59.659378 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:59.659175 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:13:03.659168907 +0000 UTC m=+251.757384075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : configmap references non-existent config key: service-ca.crt Jul 09 22:12:59.759903 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:59.759866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:12:59.760034 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:12:59.759927 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:12:59.760034 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:59.760016 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 22:12:59.760098 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:59.760059 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:12:59.760098 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:59.760073 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d94fb54-sbs7x: secret "image-registry-tls" not found Jul 09 22:12:59.760098 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:59.760085 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls podName:595f6c4e-df15-4da3-8fdf-a0cbc9cd6366 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:03.760069235 +0000 UTC m=+251.858284408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s5qhv" (UID: "595f6c4e-df15-4da3-8fdf-a0cbc9cd6366") : secret "samples-operator-tls" not found Jul 09 22:12:59.760199 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:12:59.760110 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls podName:d449e26d-5fdc-4014-864f-f1d1e89908f9 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:03.760100129 +0000 UTC m=+251.858315297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls") pod "image-registry-56d94fb54-sbs7x" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9") : secret "image-registry-tls" not found Jul 09 22:13:00.079429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:00.079343 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/0.log" Jul 09 22:13:00.079429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:00.079382 2576 generic.go:356] "Generic (PLEG): container finished" podID="e8d556ba-1556-4990-a7fa-b6bd1399ed5b" containerID="ff7bc94ac3ff890831f0f337bb52dc3b2d327ac9b4f7e7fb341b1481eaae6435" exitCode=255 Jul 09 22:13:00.079625 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:00.079444 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" event={"ID":"e8d556ba-1556-4990-a7fa-b6bd1399ed5b","Type":"ContainerDied","Data":"ff7bc94ac3ff890831f0f337bb52dc3b2d327ac9b4f7e7fb341b1481eaae6435"} Jul 09 22:13:00.079692 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:00.079678 2576 scope.go:117] "RemoveContainer" containerID="ff7bc94ac3ff890831f0f337bb52dc3b2d327ac9b4f7e7fb341b1481eaae6435" Jul 09 22:13:01.084693 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.084665 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:13:01.085159 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.085082 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/0.log" Jul 09 22:13:01.085159 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.085113 2576 generic.go:356] "Generic (PLEG): container finished" podID="e8d556ba-1556-4990-a7fa-b6bd1399ed5b" containerID="78710d8b08cb510de1031867a50bd5b2f79dd93bda6af463290ab7b7ef945aee" exitCode=255 Jul 09 22:13:01.085268 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.085149 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" event={"ID":"e8d556ba-1556-4990-a7fa-b6bd1399ed5b","Type":"ContainerDied","Data":"78710d8b08cb510de1031867a50bd5b2f79dd93bda6af463290ab7b7ef945aee"} Jul 09 22:13:01.085268 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.085221 2576 scope.go:117] "RemoveContainer" containerID="ff7bc94ac3ff890831f0f337bb52dc3b2d327ac9b4f7e7fb341b1481eaae6435" Jul 09 22:13:01.085422 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.085407 2576 scope.go:117] "RemoveContainer" containerID="78710d8b08cb510de1031867a50bd5b2f79dd93bda6af463290ab7b7ef945aee" Jul 09 22:13:01.085611 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:01.085587 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nr6vw_openshift-console-operator(e8d556ba-1556-4990-a7fa-b6bd1399ed5b)\"" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" podUID="e8d556ba-1556-4990-a7fa-b6bd1399ed5b" Jul 09 22:13:01.831298 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.831266 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kmjsh"] Jul 09 22:13:01.835438 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.835422 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:01.839660 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.839638 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 22:13:01.839801 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.839663 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 22:13:01.839801 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.839642 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 22:13:01.840895 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.840862 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 22:13:01.841004 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.840898 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lm8tn\"" Jul 09 22:13:01.849757 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.849735 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kmjsh"] Jul 09 22:13:01.977600 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.977564 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-crio-socket\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:01.977831 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.977607 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:01.977831 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.977639 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-data-volume\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:01.977831 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.977747 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bt7h\" (UniqueName: \"kubernetes.io/projected/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-api-access-2bt7h\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:01.977831 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:01.977813 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.078307 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.078258 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-crio-socket\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.078435 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.078318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.078435 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.078359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-data-volume\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.078435 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.078389 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-crio-socket\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.078435 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.078419 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bt7h\" (UniqueName: \"kubernetes.io/projected/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-api-access-2bt7h\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.078629 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.078466 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.078629 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:02.078477 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:13:02.078629 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:02.078532 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:02.578517526 +0000 UTC m=+250.676732693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : secret "insights-runtime-extractor-tls" not found Jul 09 22:13:02.078629 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:02.078550 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:13:02.078629 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:02.078618 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:02.578600644 +0000 UTC m=+250.676815824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : configmap "kube-rbac-proxy" not found Jul 09 22:13:02.078828 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.078746 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-data-volume\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.089429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.089380 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:13:02.089735 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.089686 2576 scope.go:117] "RemoveContainer" containerID="78710d8b08cb510de1031867a50bd5b2f79dd93bda6af463290ab7b7ef945aee" Jul 09 22:13:02.089844 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:02.089825 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nr6vw_openshift-console-operator(e8d556ba-1556-4990-a7fa-b6bd1399ed5b)\"" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" podUID="e8d556ba-1556-4990-a7fa-b6bd1399ed5b" Jul 09 22:13:02.090519 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.090501 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bt7h\" (UniqueName: \"kubernetes.io/projected/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-api-access-2bt7h\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.582313 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.582284 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.582497 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:02.582388 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:02.582497 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:02.582447 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:13:02.582497 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:02.582467 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:13:02.582590 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:02.582517 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:03.582499623 +0000 UTC m=+251.680714792 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : configmap "kube-rbac-proxy" not found Jul 09 22:13:02.582590 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:02.582533 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:03.582524779 +0000 UTC m=+251.680739951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : secret "insights-runtime-extractor-tls" not found Jul 09 22:13:03.590810 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:03.590782 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:03.591102 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:03.590843 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:03.591102 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.590901 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:13:03.591102 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.590962 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:05.590947327 +0000 UTC m=+253.689162495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : configmap "kube-rbac-proxy" not found Jul 09 22:13:03.591102 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.590973 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:13:03.591102 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.591016 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:05.591001286 +0000 UTC m=+253.689216454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : secret "insights-runtime-extractor-tls" not found Jul 09 22:13:03.692050 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:03.692006 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:03.692050 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:03.692055 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:03.692201 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.692155 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 22:13:03.692201 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.692155 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:13:11.692139968 +0000 UTC m=+259.790355137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : configmap references non-existent config key: service-ca.crt Jul 09 22:13:03.692201 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.692198 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:13:11.692188379 +0000 UTC m=+259.790403548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : secret "router-metrics-certs-default" not found Jul 09 22:13:03.793326 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:03.793246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:13:03.793326 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:03.793303 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:13:03.793468 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.793405 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 22:13:03.793468 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.793418 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d94fb54-sbs7x: secret "image-registry-tls" not found Jul 09 22:13:03.793468 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.793459 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls podName:d449e26d-5fdc-4014-864f-f1d1e89908f9 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:11.793446216 +0000 UTC m=+259.891661384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls") pod "image-registry-56d94fb54-sbs7x" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9") : secret "image-registry-tls" not found Jul 09 22:13:03.793564 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.793405 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 22:13:03.793564 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:03.793541 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls podName:595f6c4e-df15-4da3-8fdf-a0cbc9cd6366 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:11.793524595 +0000 UTC m=+259.891739779 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-s5qhv" (UID: "595f6c4e-df15-4da3-8fdf-a0cbc9cd6366") : secret "samples-operator-tls" not found Jul 09 22:13:03.794278 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:03.794262 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-65pnm_dff8d8d0-93d0-40a2-8061-9fb8f8903316/dns-node-resolver/0.log" Jul 09 22:13:04.094797 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:04.094758 2576 generic.go:356] "Generic (PLEG): container finished" podID="c8dc1300-7d4e-4aef-89ec-74fd5e7c9519" containerID="7fe1a5e07296e20b7eba4192bcfde65348b731f898de1c9de79c68b11ae13021" exitCode=255 Jul 09 22:13:04.094797 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:04.094799 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" event={"ID":"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519","Type":"ContainerDied","Data":"7fe1a5e07296e20b7eba4192bcfde65348b731f898de1c9de79c68b11ae13021"} Jul 09 22:13:04.095112 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:04.095093 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:13:04.095216 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:04.095201 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 22:13:04.095265 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:04.095259 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs podName:173926ea-7469-4f39-a90b-5ddaf5ec576c nodeName:}" failed. No retries permitted until 2026-07-09 22:15:06.095241782 +0000 UTC m=+374.193456954 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs") pod "network-metrics-daemon-6969j" (UID: "173926ea-7469-4f39-a90b-5ddaf5ec576c") : secret "metrics-daemon-secret" not found Jul 09 22:13:04.100190 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:04.100170 2576 scope.go:117] "RemoveContainer" containerID="7fe1a5e07296e20b7eba4192bcfde65348b731f898de1c9de79c68b11ae13021" Jul 09 22:13:04.793491 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:04.793461 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lj48t_7e53afda-97f8-4484-9c7c-b747bde49e8f/node-ca/0.log" Jul 09 22:13:05.099736 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:05.099697 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85fc4456fb-995kk" event={"ID":"c8dc1300-7d4e-4aef-89ec-74fd5e7c9519","Type":"ContainerStarted","Data":"241a0cfedb2f9add7f141a507c4a0840e35301cf36125e6e8ce57a3ac0e46a30"} Jul 09 22:13:05.608382 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:05.608347 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:05.608542 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:05.608404 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:05.608542 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:05.608466 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:13:05.608542 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:05.608493 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:13:05.608542 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:05.608531 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:09.608516214 +0000 UTC m=+257.706731386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : configmap "kube-rbac-proxy" not found Jul 09 22:13:05.608542 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:05.608545 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:09.608538829 +0000 UTC m=+257.706753996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : secret "insights-runtime-extractor-tls" not found Jul 09 22:13:05.793511 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:05.793481 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rxhvj_e252f3c5-1f99-4dd1-8d97-8d666602d58c/serve-healthcheck-canary/0.log" Jul 09 22:13:06.377188 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:06.377145 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:13:06.377350 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:06.377218 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:13:06.377521 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:06.377509 2576 scope.go:117] "RemoveContainer" containerID="78710d8b08cb510de1031867a50bd5b2f79dd93bda6af463290ab7b7ef945aee" Jul 09 22:13:06.377685 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:06.377665 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-nr6vw_openshift-console-operator(e8d556ba-1556-4990-a7fa-b6bd1399ed5b)\"" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" podUID="e8d556ba-1556-4990-a7fa-b6bd1399ed5b" Jul 09 22:13:09.640448 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:09.640410 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:09.640927 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:09.640535 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:09.640927 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:09.640565 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 22:13:09.640927 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:09.640626 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:17.64061224 +0000 UTC m=+265.738827409 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : secret "insights-runtime-extractor-tls" not found Jul 09 22:13:09.640927 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:09.640652 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 22:13:09.640927 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:09.640706 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm podName:c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f nodeName:}" failed. No retries permitted until 2026-07-09 22:13:17.64069376 +0000 UTC m=+265.738908927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kmjsh" (UID: "c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f") : configmap "kube-rbac-proxy" not found Jul 09 22:13:11.758074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:11.758038 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:11.758074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:11.758075 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:11.758491 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:11.758203 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle podName:e28edf89-6b16-4538-97f4-7f72e368a48d nodeName:}" failed. No retries permitted until 2026-07-09 22:13:27.758186946 +0000 UTC m=+275.856402118 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle") pod "router-default-58c6fd45c7-x2b2x" (UID: "e28edf89-6b16-4538-97f4-7f72e368a48d") : configmap references non-existent config key: service-ca.crt Jul 09 22:13:11.760409 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:11.760392 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e28edf89-6b16-4538-97f4-7f72e368a48d-metrics-certs\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:11.859465 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:11.859437 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:13:11.859622 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:11.859487 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:13:11.861844 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:11.861814 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") pod \"image-registry-56d94fb54-sbs7x\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:13:11.861955 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:11.861926 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/595f6c4e-df15-4da3-8fdf-a0cbc9cd6366-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-s5qhv\" (UID: \"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:13:11.967275 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:11.967227 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" Jul 09 22:13:11.987927 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:11.987902 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:13:12.094366 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:12.094341 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv"] Jul 09 22:13:12.120523 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:12.120497 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d94fb54-sbs7x"] Jul 09 22:13:12.123660 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:13:12.123637 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd449e26d_5fdc_4014_864f_f1d1e89908f9.slice/crio-1c4ed9526af4bb48a0c64a34409529384c5609726e3a8744342369cd298b9ac7 WatchSource:0}: Error finding container 1c4ed9526af4bb48a0c64a34409529384c5609726e3a8744342369cd298b9ac7: Status 404 returned error can't find the container with id 1c4ed9526af4bb48a0c64a34409529384c5609726e3a8744342369cd298b9ac7 Jul 09 22:13:13.125903 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:13.125827 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" event={"ID":"d449e26d-5fdc-4014-864f-f1d1e89908f9","Type":"ContainerStarted","Data":"6f88f454001dbe3a766dd8960bdd4d9faab2beb610d6fc2c13f4928cae7b551a"} Jul 09 22:13:13.126317 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:13.125951 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:13:13.126317 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:13.125973 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" event={"ID":"d449e26d-5fdc-4014-864f-f1d1e89908f9","Type":"ContainerStarted","Data":"1c4ed9526af4bb48a0c64a34409529384c5609726e3a8744342369cd298b9ac7"} Jul 09 22:13:13.126945 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:13.126927 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" event={"ID":"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366","Type":"ContainerStarted","Data":"be0ecd4879d3dafd5d5eaea63948e02285e1746701a21ebeab1278a91fa68664"} Jul 09 22:13:13.150367 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:13.150322 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" podStartSLOduration=17.150308163 podStartE2EDuration="17.150308163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:13.149818031 +0000 UTC m=+261.248033220" watchObservedRunningTime="2026-07-09 22:13:13.150308163 +0000 UTC m=+261.248523353" Jul 09 22:13:15.134757 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:15.134705 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" event={"ID":"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366","Type":"ContainerStarted","Data":"d685f87e65d6e9373766e0dc5056e287ffe4b86c62ca50d923c6d17eac13b583"} Jul 09 22:13:15.135132 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:15.134782 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" event={"ID":"595f6c4e-df15-4da3-8fdf-a0cbc9cd6366","Type":"ContainerStarted","Data":"8768613c967182d28fe90bbc86a3c3ccd6da78b7d9dea8c927bb82ca60ae6180"} Jul 09 22:13:15.154332 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:15.154286 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-s5qhv" podStartSLOduration=16.728594557 podStartE2EDuration="19.154269865s" totalImagesPullingTime="2.425675308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:15.154150416 +0000 UTC m=+263.252365608" watchObservedRunningTime="2026-07-09 22:13:15.154269865 +0000 UTC m=+263.252485055" Jul 09 22:13:17.708079 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:17.708042 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:17.708455 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:17.708100 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:17.708556 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:17.708536 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:17.710247 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:17.710230 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kmjsh\" (UID: \"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f\") " pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:17.743435 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:17.743408 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kmjsh" Jul 09 22:13:17.872593 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:17.870447 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kmjsh"] Jul 09 22:13:17.875419 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:13:17.875389 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ba8067_5c36_4bb1_9cb0_33ab88d95b1f.slice/crio-a2e44f2c6bb97b31bb1c6a37e231bfd969da8a6bd5a28d55f37b35f050ac6000 WatchSource:0}: Error finding container a2e44f2c6bb97b31bb1c6a37e231bfd969da8a6bd5a28d55f37b35f050ac6000: Status 404 returned error can't find the container with id a2e44f2c6bb97b31bb1c6a37e231bfd969da8a6bd5a28d55f37b35f050ac6000 Jul 09 22:13:18.144285 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:18.144247 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmjsh" event={"ID":"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f","Type":"ContainerStarted","Data":"a2e44f2c6bb97b31bb1c6a37e231bfd969da8a6bd5a28d55f37b35f050ac6000"} Jul 09 22:13:21.383032 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:21.383002 2576 scope.go:117] "RemoveContainer" containerID="78710d8b08cb510de1031867a50bd5b2f79dd93bda6af463290ab7b7ef945aee" Jul 09 22:13:22.160978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:22.160952 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:13:22.161156 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:22.161065 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" event={"ID":"e8d556ba-1556-4990-a7fa-b6bd1399ed5b","Type":"ContainerStarted","Data":"5bbce32d91b724a792331daeccc798a6b4d2a4baee944bd435a6834b90fba717"} Jul 09 22:13:22.161424 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:22.161378 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:13:22.162755 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:22.162711 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmjsh" event={"ID":"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f","Type":"ContainerStarted","Data":"568a0ca5d4870e4c95a44e1b37d401481ae4e3cce67e54d174b6602a86527faa"} Jul 09 22:13:22.181365 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:22.181323 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" podStartSLOduration=23.574754607 podStartE2EDuration="26.181309894s" totalImagesPullingTime="2.606555287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:22.180518018 +0000 UTC m=+270.278733208" watchObservedRunningTime="2026-07-09 22:13:22.181309894 +0000 UTC m=+270.279525108" Jul 09 22:13:22.567327 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:22.567294 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-nr6vw" Jul 09 22:13:23.121211 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.121179 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56d94fb54-sbs7x"] Jul 09 22:13:23.151024 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.150976 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c5d578676-dfpfv"] Jul 09 22:13:23.154810 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.154785 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.168905 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.168874 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmjsh" event={"ID":"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f","Type":"ContainerStarted","Data":"ffe18d270f74362715973f33e9230a79ce0bc57a651cef06db118e6378db4061"} Jul 09 22:13:23.169031 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.168941 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c5d578676-dfpfv"] Jul 09 22:13:23.252591 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.252556 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d6c84071-071a-4247-a8c1-00db7051e14a-registry-tls\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.252788 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.252612 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d6c84071-071a-4247-a8c1-00db7051e14a-installation-pull-secrets\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.252788 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.252642 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4htk\" (UniqueName: \"kubernetes.io/projected/d6c84071-071a-4247-a8c1-00db7051e14a-kube-api-access-k4htk\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.252788 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.252703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d6c84071-071a-4247-a8c1-00db7051e14a-registry-certificates\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.252788 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.252781 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d6c84071-071a-4247-a8c1-00db7051e14a-image-registry-private-configuration\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.253000 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.252807 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d6c84071-071a-4247-a8c1-00db7051e14a-trusted-ca\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.253000 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.252844 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d6c84071-071a-4247-a8c1-00db7051e14a-ca-trust-extracted\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.253000 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.252872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6c84071-071a-4247-a8c1-00db7051e14a-bound-sa-token\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.353387 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.353351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6c84071-071a-4247-a8c1-00db7051e14a-bound-sa-token\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.353567 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.353531 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d6c84071-071a-4247-a8c1-00db7051e14a-registry-tls\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.353753 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.353707 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d6c84071-071a-4247-a8c1-00db7051e14a-installation-pull-secrets\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.353823 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.353778 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4htk\" (UniqueName: \"kubernetes.io/projected/d6c84071-071a-4247-a8c1-00db7051e14a-kube-api-access-k4htk\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.353875 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.353859 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d6c84071-071a-4247-a8c1-00db7051e14a-registry-certificates\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.354250 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.353948 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d6c84071-071a-4247-a8c1-00db7051e14a-image-registry-private-configuration\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.354250 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.353979 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d6c84071-071a-4247-a8c1-00db7051e14a-trusted-ca\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.354250 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.354011 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d6c84071-071a-4247-a8c1-00db7051e14a-ca-trust-extracted\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.354515 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.354474 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d6c84071-071a-4247-a8c1-00db7051e14a-ca-trust-extracted\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.354884 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.354857 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d6c84071-071a-4247-a8c1-00db7051e14a-registry-certificates\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.355176 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.355153 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d6c84071-071a-4247-a8c1-00db7051e14a-trusted-ca\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.356675 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.356657 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d6c84071-071a-4247-a8c1-00db7051e14a-registry-tls\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.356863 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.356844 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d6c84071-071a-4247-a8c1-00db7051e14a-image-registry-private-configuration\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.357319 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.357304 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d6c84071-071a-4247-a8c1-00db7051e14a-installation-pull-secrets\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.365349 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.365323 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6c84071-071a-4247-a8c1-00db7051e14a-bound-sa-token\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.365834 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.365808 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4htk\" (UniqueName: \"kubernetes.io/projected/d6c84071-071a-4247-a8c1-00db7051e14a-kube-api-access-k4htk\") pod \"image-registry-c5d578676-dfpfv\" (UID: \"d6c84071-071a-4247-a8c1-00db7051e14a\") " pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.466522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.466445 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:23.623557 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:23.623523 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c5d578676-dfpfv"] Jul 09 22:13:23.626503 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:13:23.626473 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c84071_071a_4247_a8c1_00db7051e14a.slice/crio-b515694f0b80d914141eff35e64602f3af7e0a7f04a8cb10f230cefa34ef6f08 WatchSource:0}: Error finding container b515694f0b80d914141eff35e64602f3af7e0a7f04a8cb10f230cefa34ef6f08: Status 404 returned error can't find the container with id b515694f0b80d914141eff35e64602f3af7e0a7f04a8cb10f230cefa34ef6f08 Jul 09 22:13:24.174232 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:24.174190 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c5d578676-dfpfv" event={"ID":"d6c84071-071a-4247-a8c1-00db7051e14a","Type":"ContainerStarted","Data":"394747e2d7aad20ae366692882bc0807d8ac1bef7391f6e07975d17cf4d13ee2"} Jul 09 22:13:24.174404 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:24.174253 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c5d578676-dfpfv" event={"ID":"d6c84071-071a-4247-a8c1-00db7051e14a","Type":"ContainerStarted","Data":"b515694f0b80d914141eff35e64602f3af7e0a7f04a8cb10f230cefa34ef6f08"} Jul 09 22:13:24.197075 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:24.197027 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c5d578676-dfpfv" podStartSLOduration=1.197008283 podStartE2EDuration="1.197008283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:24.196014005 +0000 UTC m=+272.294229195" watchObservedRunningTime="2026-07-09 22:13:24.197008283 +0000 UTC m=+272.295223472" Jul 09 22:13:25.178794 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:25.178754 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kmjsh" event={"ID":"c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f","Type":"ContainerStarted","Data":"c6cebe2af9a3454bf5ce962c91c83fa147b0af559ee647b3081a3c6ea4d16d45"} Jul 09 22:13:25.179167 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:25.178858 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:25.208483 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:25.208390 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kmjsh" podStartSLOduration=17.254184273 podStartE2EDuration="24.208373978s" totalImagesPullingTime="6.954189705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:13:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:25.207839865 +0000 UTC m=+273.306055056" watchObservedRunningTime="2026-07-09 22:13:25.208373978 +0000 UTC m=+273.306589168" Jul 09 22:13:27.791152 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:27.791112 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:27.791759 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:27.791741 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28edf89-6b16-4538-97f4-7f72e368a48d-service-ca-bundle\") pod \"router-default-58c6fd45c7-x2b2x\" (UID: \"e28edf89-6b16-4538-97f4-7f72e368a48d\") " pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:28.088833 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:28.088800 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:28.218351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:28.218329 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58c6fd45c7-x2b2x"] Jul 09 22:13:28.220645 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:13:28.220621 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28edf89_6b16_4538_97f4_7f72e368a48d.slice/crio-c30258b287986b6c0bb006462f7253ffdaa11807abf4580133fb54c27959a323 WatchSource:0}: Error finding container c30258b287986b6c0bb006462f7253ffdaa11807abf4580133fb54c27959a323: Status 404 returned error can't find the container with id c30258b287986b6c0bb006462f7253ffdaa11807abf4580133fb54c27959a323 Jul 09 22:13:29.192902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:29.192867 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" event={"ID":"e28edf89-6b16-4538-97f4-7f72e368a48d","Type":"ContainerStarted","Data":"335bfb036714a9f7495316758a784a73b86184444aeea5ecb80616029c757a8c"} Jul 09 22:13:29.193376 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:29.192918 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" event={"ID":"e28edf89-6b16-4538-97f4-7f72e368a48d","Type":"ContainerStarted","Data":"c30258b287986b6c0bb006462f7253ffdaa11807abf4580133fb54c27959a323"} Jul 09 22:13:29.214281 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:29.214236 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" podStartSLOduration=34.214221327 podStartE2EDuration="34.214221327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:29.213798207 +0000 UTC m=+277.312013380" watchObservedRunningTime="2026-07-09 22:13:29.214221327 +0000 UTC m=+277.312436515" Jul 09 22:13:30.089904 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:30.089871 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:30.092575 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:30.092552 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:30.196315 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:30.196292 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:30.197568 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:30.197547 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-58c6fd45c7-x2b2x" Jul 09 22:13:33.126488 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:33.126450 2576 patch_prober.go:28] interesting pod/image-registry-56d94fb54-sbs7x container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 22:13:33.126941 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:33.126510 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" podUID="d449e26d-5fdc-4014-864f-f1d1e89908f9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 22:13:33.867115 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:33.867071 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dcgsk" podUID="6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c" Jul 09 22:13:34.207806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:34.207743 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dcgsk" Jul 09 22:13:37.757899 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:37.757871 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:13:37.760040 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:37.760018 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c-metrics-tls\") pod \"dns-default-dcgsk\" (UID: \"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c\") " pod="openshift-dns/dns-default-dcgsk" Jul 09 22:13:37.812450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:37.812419 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77st2\"" Jul 09 22:13:37.819412 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:37.819396 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dcgsk" Jul 09 22:13:37.965475 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:37.965455 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dcgsk"] Jul 09 22:13:37.967107 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:13:37.967081 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b605cb1_c348_4a4a_bbd6_bdfb4b34bb1c.slice/crio-1fb744898f192263b6b1f4f671d81a12a68b1cb451c30e17d56b28662f403547 WatchSource:0}: Error finding container 1fb744898f192263b6b1f4f671d81a12a68b1cb451c30e17d56b28662f403547: Status 404 returned error can't find the container with id 1fb744898f192263b6b1f4f671d81a12a68b1cb451c30e17d56b28662f403547 Jul 09 22:13:38.220190 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:38.220159 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dcgsk" event={"ID":"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c","Type":"ContainerStarted","Data":"1fb744898f192263b6b1f4f671d81a12a68b1cb451c30e17d56b28662f403547"} Jul 09 22:13:40.227713 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.227677 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dcgsk" event={"ID":"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c","Type":"ContainerStarted","Data":"f428e84e16c3b76c13d45a14b5fb0aa8ef16bad7efc04a2b0e9e337c1c73f143"} Jul 09 22:13:40.228116 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.227747 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dcgsk" event={"ID":"6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c","Type":"ContainerStarted","Data":"11a044f5c5b21f44a39bdcc2f51a3ecd6f6504a2c946c65d21fa54f81cd8bcfc"} Jul 09 22:13:40.228116 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.227829 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dcgsk" Jul 09 22:13:40.360838 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.360789 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dcgsk" podStartSLOduration=252.586371534 podStartE2EDuration="4m14.360769044s" totalImagesPullingTime="1.77439751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:40.247177807 +0000 UTC m=+288.345392998" watchObservedRunningTime="2026-07-09 22:13:40.360769044 +0000 UTC m=+288.458984235" Jul 09 22:13:40.361555 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.361539 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk"] Jul 09 22:13:40.364741 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.364707 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.368319 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.368284 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 22:13:40.368471 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.368412 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 22:13:40.368543 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.368412 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 22:13:40.368543 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.368509 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sb7k5"] Jul 09 22:13:40.368746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.368709 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fq86x\"" Jul 09 22:13:40.368897 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.368882 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 22:13:40.369553 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.369538 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 22:13:40.369621 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.369575 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 22:13:40.371896 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.371880 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.375280 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.375261 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 22:13:40.375380 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.375334 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mb9sv\"" Jul 09 22:13:40.375380 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.375344 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 22:13:40.375801 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.375784 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 22:13:40.376287 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.376272 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk"] Jul 09 22:13:40.379852 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.379827 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.379956 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.379866 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.379956 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.379893 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/187f0a72-40a2-47c9-b5ab-a203ca59ff58-root\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.379956 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.379929 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-tls\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.380236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.379965 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpplw\" (UniqueName: \"kubernetes.io/projected/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-api-access-wpplw\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.380236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380019 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/187f0a72-40a2-47c9-b5ab-a203ca59ff58-sys\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.380236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380043 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xgrp\" (UniqueName: \"kubernetes.io/projected/187f0a72-40a2-47c9-b5ab-a203ca59ff58-kube-api-access-5xgrp\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.380236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380075 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.380236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f446a306-a346-4583-9fa7-d5e505eb47d7-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.380236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380132 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-textfile\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.380236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380153 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/187f0a72-40a2-47c9-b5ab-a203ca59ff58-metrics-client-ca\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.380236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380188 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-wtmp\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.380236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380237 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.380615 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380265 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-accelerators-collector-config\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.380615 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.380300 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f446a306-a346-4583-9fa7-d5e505eb47d7-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.480862 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.480779 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.480862 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.480811 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f446a306-a346-4583-9fa7-d5e505eb47d7-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.480862 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.480834 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-textfile\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481126 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.480910 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/187f0a72-40a2-47c9-b5ab-a203ca59ff58-metrics-client-ca\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481126 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:40.480920 2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 09 22:13:40.481126 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:40.481045 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-tls podName:f446a306-a346-4583-9fa7-d5e505eb47d7 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:40.981023942 +0000 UTC m=+289.079239126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-5gtgk" (UID: "f446a306-a346-4583-9fa7-d5e505eb47d7") : secret "kube-state-metrics-tls" not found Jul 09 22:13:40.481126 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.480961 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-wtmp\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481126 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481096 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-wtmp\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481126 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481120 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481154 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-accelerators-collector-config\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481199 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f446a306-a346-4583-9fa7-d5e505eb47d7-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481204 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-textfile\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481214 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f446a306-a346-4583-9fa7-d5e505eb47d7-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481267 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481298 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481326 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/187f0a72-40a2-47c9-b5ab-a203ca59ff58-root\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481356 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-tls\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481393 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpplw\" (UniqueName: \"kubernetes.io/projected/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-api-access-wpplw\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481422 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/187f0a72-40a2-47c9-b5ab-a203ca59ff58-sys\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.481464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481447 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xgrp\" (UniqueName: \"kubernetes.io/projected/187f0a72-40a2-47c9-b5ab-a203ca59ff58-kube-api-access-5xgrp\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.482037 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481531 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/187f0a72-40a2-47c9-b5ab-a203ca59ff58-metrics-client-ca\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.482037 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481756 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-accelerators-collector-config\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.482037 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481837 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/187f0a72-40a2-47c9-b5ab-a203ca59ff58-sys\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.482037 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481878 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/187f0a72-40a2-47c9-b5ab-a203ca59ff58-root\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.482037 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481971 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f446a306-a346-4583-9fa7-d5e505eb47d7-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.482037 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.481983 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.483623 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.483595 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.483766 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.483700 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.483766 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.483752 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/187f0a72-40a2-47c9-b5ab-a203ca59ff58-node-exporter-tls\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.491754 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.491712 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xgrp\" (UniqueName: \"kubernetes.io/projected/187f0a72-40a2-47c9-b5ab-a203ca59ff58-kube-api-access-5xgrp\") pod \"node-exporter-sb7k5\" (UID: \"187f0a72-40a2-47c9-b5ab-a203ca59ff58\") " pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.492413 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.492395 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpplw\" (UniqueName: \"kubernetes.io/projected/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-api-access-wpplw\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.682933 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.682900 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sb7k5" Jul 09 22:13:40.694176 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:13:40.694145 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod187f0a72_40a2_47c9_b5ab_a203ca59ff58.slice/crio-419f5de8579cf669924cee2c404c92943423e8de2a6a4522192623d64a47c1c8 WatchSource:0}: Error finding container 419f5de8579cf669924cee2c404c92943423e8de2a6a4522192623d64a47c1c8: Status 404 returned error can't find the container with id 419f5de8579cf669924cee2c404c92943423e8de2a6a4522192623d64a47c1c8 Jul 09 22:13:40.985627 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:40.985591 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:40.985804 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:40.985713 2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 09 22:13:40.985804 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:40.985792 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-tls podName:f446a306-a346-4583-9fa7-d5e505eb47d7 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:41.985778164 +0000 UTC m=+290.083993333 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-5gtgk" (UID: "f446a306-a346-4583-9fa7-d5e505eb47d7") : secret "kube-state-metrics-tls" not found Jul 09 22:13:41.233771 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:41.233730 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sb7k5" event={"ID":"187f0a72-40a2-47c9-b5ab-a203ca59ff58","Type":"ContainerStarted","Data":"419f5de8579cf669924cee2c404c92943423e8de2a6a4522192623d64a47c1c8"} Jul 09 22:13:41.992583 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:41.992501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:41.994761 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:41.994731 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f446a306-a346-4583-9fa7-d5e505eb47d7-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5gtgk\" (UID: \"f446a306-a346-4583-9fa7-d5e505eb47d7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:42.174016 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:42.173983 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" Jul 09 22:13:42.239993 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:42.239959 2576 generic.go:356] "Generic (PLEG): container finished" podID="187f0a72-40a2-47c9-b5ab-a203ca59ff58" containerID="5a009a4791c5024ad2e1703ba5536ed3193eba1e1fb7a9e01aafc83c6743869b" exitCode=0 Jul 09 22:13:42.240314 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:42.239999 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sb7k5" event={"ID":"187f0a72-40a2-47c9-b5ab-a203ca59ff58","Type":"ContainerDied","Data":"5a009a4791c5024ad2e1703ba5536ed3193eba1e1fb7a9e01aafc83c6743869b"} Jul 09 22:13:42.306016 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:42.305902 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk"] Jul 09 22:13:42.308207 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:13:42.308180 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf446a306_a346_4583_9fa7_d5e505eb47d7.slice/crio-d976a898eaaddeab1cdedc388e09154cfa00f814985fcc8c635a785158d88b2a WatchSource:0}: Error finding container d976a898eaaddeab1cdedc388e09154cfa00f814985fcc8c635a785158d88b2a: Status 404 returned error can't find the container with id d976a898eaaddeab1cdedc388e09154cfa00f814985fcc8c635a785158d88b2a Jul 09 22:13:43.126669 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.126635 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:13:43.248432 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.248395 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sb7k5" event={"ID":"187f0a72-40a2-47c9-b5ab-a203ca59ff58","Type":"ContainerStarted","Data":"8ddfda129bdb58a18856f808f1c00cb77b8cc1cd587d9890069f1a8b24311cf3"} Jul 09 22:13:43.248913 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.248460 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sb7k5" event={"ID":"187f0a72-40a2-47c9-b5ab-a203ca59ff58","Type":"ContainerStarted","Data":"4cf796001ec64327011bcc04be9602bd78b61595337c12b8b80c27bbb0332a08"} Jul 09 22:13:43.250007 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.249980 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" event={"ID":"f446a306-a346-4583-9fa7-d5e505eb47d7","Type":"ContainerStarted","Data":"d976a898eaaddeab1cdedc388e09154cfa00f814985fcc8c635a785158d88b2a"} Jul 09 22:13:43.272436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.272389 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sb7k5" podStartSLOduration=2.172405312 podStartE2EDuration="3.272374274s" totalImagesPullingTime="1.037561001s" totalInitContainerRuntime="62.407961ms" isStatefulPod=true podCreationTimestamp="2026-07-09 22:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:43.271176869 +0000 UTC m=+291.369392059" watchObservedRunningTime="2026-07-09 22:13:43.272374274 +0000 UTC m=+291.370589464" Jul 09 22:13:43.336963 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.336930 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6bcd449898-8rg6h"] Jul 09 22:13:43.340757 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.340736 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.344540 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.344287 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 22:13:43.344540 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.344355 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 22:13:43.344540 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.344286 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 22:13:43.344540 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.344426 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 22:13:43.344540 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.344355 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-aa7pulemm8v4l\"" Jul 09 22:13:43.344540 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.344462 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 22:13:43.344540 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.344511 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-blbvs\"" Jul 09 22:13:43.352267 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.352235 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bcd449898-8rg6h"] Jul 09 22:13:43.404609 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.404540 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.404609 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.404593 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r475\" (UniqueName: \"kubernetes.io/projected/52ce14be-4c7b-49f3-b018-8841249939c9-kube-api-access-7r475\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.404843 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.404688 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.404843 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.404744 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52ce14be-4c7b-49f3-b018-8841249939c9-metrics-client-ca\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.404843 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.404780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-grpc-tls\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.404843 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.404807 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.405012 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.404851 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.405012 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.404887 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-tls\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.472984 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.472934 2576 patch_prober.go:28] interesting pod/image-registry-c5d578676-dfpfv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 09 22:13:43.473138 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.472991 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c5d578676-dfpfv" podUID="d6c84071-071a-4247-a8c1-00db7051e14a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 22:13:43.505528 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.505499 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r475\" (UniqueName: \"kubernetes.io/projected/52ce14be-4c7b-49f3-b018-8841249939c9-kube-api-access-7r475\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.505691 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.505558 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.505691 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.505582 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52ce14be-4c7b-49f3-b018-8841249939c9-metrics-client-ca\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.505691 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.505607 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-grpc-tls\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.505886 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.505749 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.505886 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.505807 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.505886 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.505842 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-tls\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.506046 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.505896 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.506436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.506389 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52ce14be-4c7b-49f3-b018-8841249939c9-metrics-client-ca\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.508704 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.508654 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.508844 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.508705 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.508844 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.508761 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-grpc-tls\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.509247 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.509038 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.509446 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.509420 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-tls\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.509603 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.509584 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/52ce14be-4c7b-49f3-b018-8841249939c9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.514258 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.514237 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r475\" (UniqueName: \"kubernetes.io/projected/52ce14be-4c7b-49f3-b018-8841249939c9-kube-api-access-7r475\") pod \"thanos-querier-6bcd449898-8rg6h\" (UID: \"52ce14be-4c7b-49f3-b018-8841249939c9\") " pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.652798 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.652761 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:43.792269 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:43.792240 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bcd449898-8rg6h"] Jul 09 22:13:43.795560 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:13:43.795533 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ce14be_4c7b_49f3_b018_8841249939c9.slice/crio-b05c2aa5801d758be86d9ffd615d98c8ce21e43c1add2e763a8dd8f60d1ec0a7 WatchSource:0}: Error finding container b05c2aa5801d758be86d9ffd615d98c8ce21e43c1add2e763a8dd8f60d1ec0a7: Status 404 returned error can't find the container with id b05c2aa5801d758be86d9ffd615d98c8ce21e43c1add2e763a8dd8f60d1ec0a7 Jul 09 22:13:44.258388 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:44.258282 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" event={"ID":"f446a306-a346-4583-9fa7-d5e505eb47d7","Type":"ContainerStarted","Data":"1ac06cda96319fcbce375de26c9df6e1a24cc062b089b17719127cead039ac98"} Jul 09 22:13:44.258388 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:44.258357 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" event={"ID":"f446a306-a346-4583-9fa7-d5e505eb47d7","Type":"ContainerStarted","Data":"301fee81abc8abef5bc8472ce2283487bbf69f90d050ee823b4d828486d0c57e"} Jul 09 22:13:44.259096 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:44.258409 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" event={"ID":"f446a306-a346-4583-9fa7-d5e505eb47d7","Type":"ContainerStarted","Data":"4beee4f34f40ca0561610c43d65559c9fd8d39a6903bcba8d8573e41f0ddc602"} Jul 09 22:13:44.259690 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:44.259659 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" event={"ID":"52ce14be-4c7b-49f3-b018-8841249939c9","Type":"ContainerStarted","Data":"b05c2aa5801d758be86d9ffd615d98c8ce21e43c1add2e763a8dd8f60d1ec0a7"} Jul 09 22:13:44.276839 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:44.276790 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5gtgk" podStartSLOduration=2.87006405 podStartE2EDuration="4.276775699s" totalImagesPullingTime="1.406711649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:44.276382069 +0000 UTC m=+292.374597259" watchObservedRunningTime="2026-07-09 22:13:44.276775699 +0000 UTC m=+292.374990888" Jul 09 22:13:46.186405 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.186378 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c5d578676-dfpfv" Jul 09 22:13:46.268887 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.268852 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" event={"ID":"52ce14be-4c7b-49f3-b018-8841249939c9","Type":"ContainerStarted","Data":"6c368e17e1f27a03841abd6f3e3432ff38e4f540f232961fa87ce0fdf443cdcc"} Jul 09 22:13:46.269271 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.268917 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" event={"ID":"52ce14be-4c7b-49f3-b018-8841249939c9","Type":"ContainerStarted","Data":"560a709b325eb2e6522410a7aeaee8b90a140679f8b6964edd5c185082310817"} Jul 09 22:13:46.269271 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.268942 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" event={"ID":"52ce14be-4c7b-49f3-b018-8841249939c9","Type":"ContainerStarted","Data":"3a74906d8e623032d93d60dc1413ec9456564da9091935ae8c2454d656ff625f"} Jul 09 22:13:46.583305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.583277 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 22:13:46.586919 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.586903 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.590486 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.590459 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 22:13:46.590577 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.590463 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 22:13:46.590775 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.590467 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 22:13:46.590844 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.590473 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 22:13:46.591122 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.591105 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wcbsl\"" Jul 09 22:13:46.591187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.591145 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 22:13:46.591187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.591150 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 22:13:46.591187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.591175 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 22:13:46.591341 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.591227 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 22:13:46.592236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.592221 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 22:13:46.593138 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.593120 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 22:13:46.594393 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.594366 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8lqgvipckaonm\"" Jul 09 22:13:46.594480 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.594371 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 22:13:46.594480 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.594371 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 22:13:46.597451 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.597429 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 22:13:46.599003 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.598987 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 22:13:46.599607 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.599593 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 22:13:46.608137 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.608118 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 22:13:46.631536 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631514 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631656 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631540 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdj68\" (UniqueName: \"kubernetes.io/projected/a910b6d9-c228-481d-8242-d630fab89363-kube-api-access-tdj68\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631656 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631567 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631656 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631591 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a910b6d9-c228-481d-8242-d630fab89363-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631656 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631610 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631675 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-config\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631734 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.631874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631861 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-web-config\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.632098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631928 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.632098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631955 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.632098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.631977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.632098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.632000 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.632098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.632020 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a910b6d9-c228-481d-8242-d630fab89363-config-out\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.632098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.632037 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.632098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.632057 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.632098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.632077 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.632098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.632100 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.733486 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733449 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a910b6d9-c228-481d-8242-d630fab89363-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.733486 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733487 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.733678 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733517 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-config\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.733678 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.733678 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733658 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.733822 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733684 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.733878 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733840 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.733930 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733890 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-web-config\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.733977 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733953 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734031 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.733989 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734031 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734019 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734127 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734052 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734127 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734090 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a910b6d9-c228-481d-8242-d630fab89363-config-out\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734127 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734115 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734265 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734138 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734265 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734162 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734265 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734192 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734265 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734231 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734265 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734249 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734483 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734255 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdj68\" (UniqueName: \"kubernetes.io/projected/a910b6d9-c228-481d-8242-d630fab89363-kube-api-access-tdj68\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734483 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734326 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734483 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734338 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.734483 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.734453 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.735375 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.735064 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.737423 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.736897 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-config\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.737423 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.737077 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a910b6d9-c228-481d-8242-d630fab89363-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.737423 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.737087 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a910b6d9-c228-481d-8242-d630fab89363-config-out\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.737423 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.737093 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.737423 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.737355 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.737423 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.737362 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.737837 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.737545 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.738101 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.738079 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-web-config\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.738249 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.738093 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.738528 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.738505 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a910b6d9-c228-481d-8242-d630fab89363-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.739069 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.739046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.739666 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.739622 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.739786 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.739645 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.739844 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.739791 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.739844 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.739803 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a910b6d9-c228-481d-8242-d630fab89363-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.744362 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.744341 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdj68\" (UniqueName: \"kubernetes.io/projected/a910b6d9-c228-481d-8242-d630fab89363-kube-api-access-tdj68\") pod \"prometheus-k8s-0\" (UID: \"a910b6d9-c228-481d-8242-d630fab89363\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:46.896935 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:46.896816 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:13:47.070227 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:47.070198 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 22:13:47.074411 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:13:47.074381 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda910b6d9_c228_481d_8242_d630fab89363.slice/crio-7f74ae9f7b4bef8bee35a2836282b240ca5d03e85a70538a6fa8f106a058097f WatchSource:0}: Error finding container 7f74ae9f7b4bef8bee35a2836282b240ca5d03e85a70538a6fa8f106a058097f: Status 404 returned error can't find the container with id 7f74ae9f7b4bef8bee35a2836282b240ca5d03e85a70538a6fa8f106a058097f Jul 09 22:13:47.273382 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:47.273342 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a910b6d9-c228-481d-8242-d630fab89363","Type":"ContainerStarted","Data":"7f74ae9f7b4bef8bee35a2836282b240ca5d03e85a70538a6fa8f106a058097f"} Jul 09 22:13:47.275902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:47.275871 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" event={"ID":"52ce14be-4c7b-49f3-b018-8841249939c9","Type":"ContainerStarted","Data":"b21e8fa70f99df45f4b3131344672baf280eeb51df8812c512a397d6dae28d96"} Jul 09 22:13:47.276006 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:47.275925 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" event={"ID":"52ce14be-4c7b-49f3-b018-8841249939c9","Type":"ContainerStarted","Data":"b513eb8c99b9fa0de7dc2c9a603e82b3915240558154caf5439978d51e01a5d7"} Jul 09 22:13:47.276006 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:47.275959 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" event={"ID":"52ce14be-4c7b-49f3-b018-8841249939c9","Type":"ContainerStarted","Data":"c5d2982950202bdd247e8392b94c4a8f555b3be777825490d4090b117defab14"} Jul 09 22:13:47.276067 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:47.276038 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:47.297534 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:47.297488 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" podStartSLOduration=1.297340282 podStartE2EDuration="4.297475213s" totalImagesPullingTime="3.000134931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:47.296778621 +0000 UTC m=+295.394993811" watchObservedRunningTime="2026-07-09 22:13:47.297475213 +0000 UTC m=+295.395690402" Jul 09 22:13:48.150178 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.150110 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" podUID="d449e26d-5fdc-4014-864f-f1d1e89908f9" containerName="registry" containerID="cri-o://6f88f454001dbe3a766dd8960bdd4d9faab2beb610d6fc2c13f4928cae7b551a" gracePeriod=30 Jul 09 22:13:48.281540 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.281514 2576 generic.go:356] "Generic (PLEG): container finished" podID="d449e26d-5fdc-4014-864f-f1d1e89908f9" containerID="6f88f454001dbe3a766dd8960bdd4d9faab2beb610d6fc2c13f4928cae7b551a" exitCode=0 Jul 09 22:13:48.281877 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.281592 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" event={"ID":"d449e26d-5fdc-4014-864f-f1d1e89908f9","Type":"ContainerDied","Data":"6f88f454001dbe3a766dd8960bdd4d9faab2beb610d6fc2c13f4928cae7b551a"} Jul 09 22:13:48.406968 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.406905 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:13:48.448976 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.448944 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d449e26d-5fdc-4014-864f-f1d1e89908f9-ca-trust-extracted\") pod \"d449e26d-5fdc-4014-864f-f1d1e89908f9\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " Jul 09 22:13:48.449130 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.448987 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-image-registry-private-configuration\") pod \"d449e26d-5fdc-4014-864f-f1d1e89908f9\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " Jul 09 22:13:48.449130 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.449017 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qcr6\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-kube-api-access-6qcr6\") pod \"d449e26d-5fdc-4014-864f-f1d1e89908f9\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " Jul 09 22:13:48.449130 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.449048 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") pod \"d449e26d-5fdc-4014-864f-f1d1e89908f9\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " Jul 09 22:13:48.449130 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.449074 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-certificates\") pod \"d449e26d-5fdc-4014-864f-f1d1e89908f9\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " Jul 09 22:13:48.449342 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.449227 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-installation-pull-secrets\") pod \"d449e26d-5fdc-4014-864f-f1d1e89908f9\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " Jul 09 22:13:48.449342 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.449288 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-bound-sa-token\") pod \"d449e26d-5fdc-4014-864f-f1d1e89908f9\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " Jul 09 22:13:48.449342 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.449321 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-trusted-ca\") pod \"d449e26d-5fdc-4014-864f-f1d1e89908f9\" (UID: \"d449e26d-5fdc-4014-864f-f1d1e89908f9\") " Jul 09 22:13:48.449813 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.449525 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d449e26d-5fdc-4014-864f-f1d1e89908f9" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:13:48.449813 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.449646 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-certificates\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:13:48.450052 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.450029 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d449e26d-5fdc-4014-864f-f1d1e89908f9" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:13:48.452375 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.452291 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d449e26d-5fdc-4014-864f-f1d1e89908f9" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:13:48.452375 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.452291 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d449e26d-5fdc-4014-864f-f1d1e89908f9" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:13:48.452375 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.452173 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-kube-api-access-6qcr6" (OuterVolumeSpecName: "kube-api-access-6qcr6") pod "d449e26d-5fdc-4014-864f-f1d1e89908f9" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9"). InnerVolumeSpecName "kube-api-access-6qcr6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:13:48.452375 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.452339 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d449e26d-5fdc-4014-864f-f1d1e89908f9" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:13:48.452624 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.452429 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d449e26d-5fdc-4014-864f-f1d1e89908f9" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:13:48.458420 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.458390 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d449e26d-5fdc-4014-864f-f1d1e89908f9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d449e26d-5fdc-4014-864f-f1d1e89908f9" (UID: "d449e26d-5fdc-4014-864f-f1d1e89908f9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:13:48.550079 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.550039 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-installation-pull-secrets\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:13:48.550079 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.550069 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-bound-sa-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:13:48.550079 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.550078 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d449e26d-5fdc-4014-864f-f1d1e89908f9-trusted-ca\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:13:48.550079 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.550086 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d449e26d-5fdc-4014-864f-f1d1e89908f9-ca-trust-extracted\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:13:48.550329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.550096 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d449e26d-5fdc-4014-864f-f1d1e89908f9-image-registry-private-configuration\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:13:48.550329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.550106 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qcr6\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-kube-api-access-6qcr6\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:13:48.550329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:48.550115 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d449e26d-5fdc-4014-864f-f1d1e89908f9-registry-tls\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:13:49.286083 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:49.286052 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" Jul 09 22:13:49.286519 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:49.286051 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d94fb54-sbs7x" event={"ID":"d449e26d-5fdc-4014-864f-f1d1e89908f9","Type":"ContainerDied","Data":"1c4ed9526af4bb48a0c64a34409529384c5609726e3a8744342369cd298b9ac7"} Jul 09 22:13:49.286519 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:49.286188 2576 scope.go:117] "RemoveContainer" containerID="6f88f454001dbe3a766dd8960bdd4d9faab2beb610d6fc2c13f4928cae7b551a" Jul 09 22:13:49.287577 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:49.287552 2576 generic.go:356] "Generic (PLEG): container finished" podID="a910b6d9-c228-481d-8242-d630fab89363" containerID="f0cd9b97869411ba86fb1b6622d6743d4bd5a488742e289e4dd62c55a159c475" exitCode=0 Jul 09 22:13:49.287652 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:49.287606 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a910b6d9-c228-481d-8242-d630fab89363","Type":"ContainerDied","Data":"f0cd9b97869411ba86fb1b6622d6743d4bd5a488742e289e4dd62c55a159c475"} Jul 09 22:13:49.332402 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:49.332371 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56d94fb54-sbs7x"] Jul 09 22:13:49.336298 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:49.336276 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56d94fb54-sbs7x"] Jul 09 22:13:49.384164 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:13:49.384129 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6969j" podUID="173926ea-7469-4f39-a90b-5ddaf5ec576c" Jul 09 22:13:50.236692 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:50.236659 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dcgsk" Jul 09 22:13:50.387177 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:50.387142 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d449e26d-5fdc-4014-864f-f1d1e89908f9" path="/var/lib/kubelet/pods/d449e26d-5fdc-4014-864f-f1d1e89908f9/volumes" Jul 09 22:13:52.767954 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:52.767929 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:13:52.768451 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:52.768381 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:13:52.777647 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:52.777625 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 22:13:53.288053 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:53.287975 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6bcd449898-8rg6h" Jul 09 22:13:53.306829 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:53.306796 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a910b6d9-c228-481d-8242-d630fab89363","Type":"ContainerStarted","Data":"f9e8c8f7623d133842f762ad6e89a460785ee3f60fb85a53aac7e7dbe9729266"} Jul 09 22:13:53.306975 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:53.306869 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a910b6d9-c228-481d-8242-d630fab89363","Type":"ContainerStarted","Data":"ed795fae5f6744582426a1202c1861be357204121b96f3fee1779f0d4df78134"} Jul 09 22:13:53.306975 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:53.306909 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a910b6d9-c228-481d-8242-d630fab89363","Type":"ContainerStarted","Data":"e6f250fa323667285c02eeaaba425dcd2b4381c99beb101380e510b753f7e8eb"} Jul 09 22:13:53.306975 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:53.306948 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a910b6d9-c228-481d-8242-d630fab89363","Type":"ContainerStarted","Data":"543bbada944512a3939d24c512a1f654d7b599936d2354dd912ccae27e0c3521"} Jul 09 22:13:53.307078 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:53.306990 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a910b6d9-c228-481d-8242-d630fab89363","Type":"ContainerStarted","Data":"158d4bc64955faa2c9cc993647b8e3c573f4490d06eb6b5a9faea62dd7e204e4"} Jul 09 22:13:53.307078 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:53.307017 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a910b6d9-c228-481d-8242-d630fab89363","Type":"ContainerStarted","Data":"086c22b1414a5fd7f2f793747d6aa48e6903d447b34fb0e6dae9ba5fce1de7bb"} Jul 09 22:13:53.343036 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:53.342983 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.707275079 podStartE2EDuration="7.342965902s" totalImagesPullingTime="4.612653948s" totalInitContainerRuntime="23.036875ms" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:53.341146722 +0000 UTC m=+301.439361956" watchObservedRunningTime="2026-07-09 22:13:53.342965902 +0000 UTC m=+301.441181090" Jul 09 22:13:56.897904 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:13:56.897871 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:14:03.382781 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:14:03.382734 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:14:46.897652 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:14:46.897564 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:14:46.912807 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:14:46.912784 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:14:47.515003 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:14:47.514975 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 22:15:06.124284 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:15:06.124241 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:15:06.126629 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:15:06.126606 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/173926ea-7469-4f39-a90b-5ddaf5ec576c-metrics-certs\") pod \"network-metrics-daemon-6969j\" (UID: \"173926ea-7469-4f39-a90b-5ddaf5ec576c\") " pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:15:06.387500 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:15:06.387427 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zm9fl\"" Jul 09 22:15:06.394291 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:15:06.394277 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6969j" Jul 09 22:15:06.523780 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:15:06.523748 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6969j"] Jul 09 22:15:06.526762 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:15:06.526734 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173926ea_7469_4f39_a90b_5ddaf5ec576c.slice/crio-b6e593c8a4b3cbfb531c6769af075826fcf8cf2b07f549c89f1a747017dc85b7 WatchSource:0}: Error finding container b6e593c8a4b3cbfb531c6769af075826fcf8cf2b07f549c89f1a747017dc85b7: Status 404 returned error can't find the container with id b6e593c8a4b3cbfb531c6769af075826fcf8cf2b07f549c89f1a747017dc85b7 Jul 09 22:15:06.528645 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:15:06.528628 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:15:06.565899 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:15:06.565867 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6969j" event={"ID":"173926ea-7469-4f39-a90b-5ddaf5ec576c","Type":"ContainerStarted","Data":"b6e593c8a4b3cbfb531c6769af075826fcf8cf2b07f549c89f1a747017dc85b7"} Jul 09 22:15:08.575556 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:15:08.575516 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6969j" event={"ID":"173926ea-7469-4f39-a90b-5ddaf5ec576c","Type":"ContainerStarted","Data":"ab0c2451745099e68b9f2d0623878783598902874327c379dd06b4528f9e5143"} Jul 09 22:15:08.575964 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:15:08.575568 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6969j" event={"ID":"173926ea-7469-4f39-a90b-5ddaf5ec576c","Type":"ContainerStarted","Data":"5e297ed9291df4bc772598088a216b9c8ed307ae4a3a9e245b8ca2a2db105f92"} Jul 09 22:17:22.744475 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.744423 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6969j" podStartSLOduration=509.614253021 podStartE2EDuration="8m30.744408757s" totalImagesPullingTime="1.130155736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:08.594090856 +0000 UTC m=+376.692306045" watchObservedRunningTime="2026-07-09 22:17:22.744408757 +0000 UTC m=+510.842623947" Jul 09 22:17:22.745485 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.745462 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm"] Jul 09 22:17:22.745822 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.745805 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d449e26d-5fdc-4014-864f-f1d1e89908f9" containerName="registry" Jul 09 22:17:22.745907 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.745824 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d449e26d-5fdc-4014-864f-f1d1e89908f9" containerName="registry" Jul 09 22:17:22.745907 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.745899 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d449e26d-5fdc-4014-864f-f1d1e89908f9" containerName="registry" Jul 09 22:17:22.748987 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.748966 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:22.759461 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.759444 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 22:17:22.759548 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.759530 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kj66l\"" Jul 09 22:17:22.759857 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.759844 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 22:17:22.760211 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.760197 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 22:17:22.791420 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.791400 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 22:17:22.798762 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.798741 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm"] Jul 09 22:17:22.920976 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.920936 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0fdc908-ab18-4b60-9d33-8e272264b19e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:22.921157 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.921009 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj96p\" (UniqueName: \"kubernetes.io/projected/a0fdc908-ab18-4b60-9d33-8e272264b19e-kube-api-access-lj96p\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:22.921157 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.921041 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a0fdc908-ab18-4b60-9d33-8e272264b19e-manifests\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:22.921285 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.921154 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a0fdc908-ab18-4b60-9d33-8e272264b19e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:22.921285 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:22.921224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0fdc908-ab18-4b60-9d33-8e272264b19e-tmp\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.021713 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.021627 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0fdc908-ab18-4b60-9d33-8e272264b19e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.021713 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.021677 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj96p\" (UniqueName: \"kubernetes.io/projected/a0fdc908-ab18-4b60-9d33-8e272264b19e-kube-api-access-lj96p\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.021713 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.021695 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a0fdc908-ab18-4b60-9d33-8e272264b19e-manifests\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.022002 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.021889 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a0fdc908-ab18-4b60-9d33-8e272264b19e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.022002 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.021945 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0fdc908-ab18-4b60-9d33-8e272264b19e-tmp\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.022110 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.022095 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a0fdc908-ab18-4b60-9d33-8e272264b19e-manifests\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.024115 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.024092 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0fdc908-ab18-4b60-9d33-8e272264b19e-tmp\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.024257 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.024129 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0fdc908-ab18-4b60-9d33-8e272264b19e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.024317 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.024295 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a0fdc908-ab18-4b60-9d33-8e272264b19e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.038221 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.038195 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj96p\" (UniqueName: \"kubernetes.io/projected/a0fdc908-ab18-4b60-9d33-8e272264b19e-kube-api-access-lj96p\") pod \"opendatahub-operator-controller-manager-5b79867d65-kt8xm\" (UID: \"a0fdc908-ab18-4b60-9d33-8e272264b19e\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.058096 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.058075 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:23.224996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:23.224965 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm"] Jul 09 22:17:23.229064 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:17:23.229043 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0fdc908_ab18_4b60_9d33_8e272264b19e.slice/crio-411902aa10cb61cf782ece150d82f7a5cee397d606a39f18cd102ff3dd29bf17 WatchSource:0}: Error finding container 411902aa10cb61cf782ece150d82f7a5cee397d606a39f18cd102ff3dd29bf17: Status 404 returned error can't find the container with id 411902aa10cb61cf782ece150d82f7a5cee397d606a39f18cd102ff3dd29bf17 Jul 09 22:17:24.075237 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:24.075202 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" event={"ID":"a0fdc908-ab18-4b60-9d33-8e272264b19e","Type":"ContainerStarted","Data":"411902aa10cb61cf782ece150d82f7a5cee397d606a39f18cd102ff3dd29bf17"} Jul 09 22:17:27.089678 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:27.089646 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" event={"ID":"a0fdc908-ab18-4b60-9d33-8e272264b19e","Type":"ContainerStarted","Data":"8b4f8300102da97744d933e8603a91d718a5a07b7b9cec31e4dcf28780168a1f"} Jul 09 22:17:28.094521 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:28.094491 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0fdc908-ab18-4b60-9d33-8e272264b19e" containerID="8b4f8300102da97744d933e8603a91d718a5a07b7b9cec31e4dcf28780168a1f" exitCode=0 Jul 09 22:17:28.094911 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:28.094539 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" event={"ID":"a0fdc908-ab18-4b60-9d33-8e272264b19e","Type":"ContainerDied","Data":"8b4f8300102da97744d933e8603a91d718a5a07b7b9cec31e4dcf28780168a1f"} Jul 09 22:17:29.100691 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:29.100650 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" event={"ID":"a0fdc908-ab18-4b60-9d33-8e272264b19e","Type":"ContainerStarted","Data":"37f6f9fcf4d00937a3aff2531494f1f4950ac74156276474378ca6f0aaec51e5"} Jul 09 22:17:29.101083 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:29.100759 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:29.142251 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:29.142201 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" podStartSLOduration=2.773602441 podStartE2EDuration="7.142185632s" totalImagesPullingTime="3.360764682s" totalInitContainerRuntime="1.007818509s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:17:29.141835074 +0000 UTC m=+517.240050256" watchObservedRunningTime="2026-07-09 22:17:29.142185632 +0000 UTC m=+517.240400821" Jul 09 22:17:40.107978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:40.107949 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-kt8xm" Jul 09 22:17:43.261619 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.261584 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x"] Jul 09 22:17:43.273501 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.273478 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.277560 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.277540 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 22:17:43.277768 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.277756 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 22:17:43.325627 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.325595 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x"] Jul 09 22:17:43.382503 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.382470 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24b4b\" (UniqueName: \"kubernetes.io/projected/c7605dd3-5b80-465f-912f-19b91742c54b-kube-api-access-24b4b\") pod \"kube-auth-proxy-7ff5d987cc-rf44x\" (UID: \"c7605dd3-5b80-465f-912f-19b91742c54b\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.382657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.382522 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7605dd3-5b80-465f-912f-19b91742c54b-tls-certs\") pod \"kube-auth-proxy-7ff5d987cc-rf44x\" (UID: \"c7605dd3-5b80-465f-912f-19b91742c54b\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.382657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.382581 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7605dd3-5b80-465f-912f-19b91742c54b-tmp\") pod \"kube-auth-proxy-7ff5d987cc-rf44x\" (UID: \"c7605dd3-5b80-465f-912f-19b91742c54b\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.484035 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.484002 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7605dd3-5b80-465f-912f-19b91742c54b-tmp\") pod \"kube-auth-proxy-7ff5d987cc-rf44x\" (UID: \"c7605dd3-5b80-465f-912f-19b91742c54b\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.484200 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.484049 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24b4b\" (UniqueName: \"kubernetes.io/projected/c7605dd3-5b80-465f-912f-19b91742c54b-kube-api-access-24b4b\") pod \"kube-auth-proxy-7ff5d987cc-rf44x\" (UID: \"c7605dd3-5b80-465f-912f-19b91742c54b\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.484200 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.484089 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7605dd3-5b80-465f-912f-19b91742c54b-tls-certs\") pod \"kube-auth-proxy-7ff5d987cc-rf44x\" (UID: \"c7605dd3-5b80-465f-912f-19b91742c54b\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.486297 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.486261 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7605dd3-5b80-465f-912f-19b91742c54b-tmp\") pod \"kube-auth-proxy-7ff5d987cc-rf44x\" (UID: \"c7605dd3-5b80-465f-912f-19b91742c54b\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.486477 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.486460 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7605dd3-5b80-465f-912f-19b91742c54b-tls-certs\") pod \"kube-auth-proxy-7ff5d987cc-rf44x\" (UID: \"c7605dd3-5b80-465f-912f-19b91742c54b\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.543548 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.543479 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24b4b\" (UniqueName: \"kubernetes.io/projected/c7605dd3-5b80-465f-912f-19b91742c54b-kube-api-access-24b4b\") pod \"kube-auth-proxy-7ff5d987cc-rf44x\" (UID: \"c7605dd3-5b80-465f-912f-19b91742c54b\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.582987 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.582956 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" Jul 09 22:17:43.640476 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.640444 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b5p8s"] Jul 09 22:17:43.644680 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.644654 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:43.652630 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.652608 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 22:17:43.653603 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.653580 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wfqmw\"" Jul 09 22:17:43.681297 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.681266 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b5p8s"] Jul 09 22:17:43.686917 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.686753 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4481c981-a256-41e3-a1dc-23eddfe664a4-cert\") pod \"odh-model-controller-7998c67dc4-b5p8s\" (UID: \"4481c981-a256-41e3-a1dc-23eddfe664a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:43.686917 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.686838 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5crfx\" (UniqueName: \"kubernetes.io/projected/4481c981-a256-41e3-a1dc-23eddfe664a4-kube-api-access-5crfx\") pod \"odh-model-controller-7998c67dc4-b5p8s\" (UID: \"4481c981-a256-41e3-a1dc-23eddfe664a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:43.767971 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:17:43.767938 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7605dd3_5b80_465f_912f_19b91742c54b.slice/crio-7c5be102665b53806fa68f963a86e1d80e7704ef674cee824a39170af3c39583 WatchSource:0}: Error finding container 7c5be102665b53806fa68f963a86e1d80e7704ef674cee824a39170af3c39583: Status 404 returned error can't find the container with id 7c5be102665b53806fa68f963a86e1d80e7704ef674cee824a39170af3c39583 Jul 09 22:17:43.771573 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.771552 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x"] Jul 09 22:17:43.787500 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.787466 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5crfx\" (UniqueName: \"kubernetes.io/projected/4481c981-a256-41e3-a1dc-23eddfe664a4-kube-api-access-5crfx\") pod \"odh-model-controller-7998c67dc4-b5p8s\" (UID: \"4481c981-a256-41e3-a1dc-23eddfe664a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:43.787607 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.787570 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4481c981-a256-41e3-a1dc-23eddfe664a4-cert\") pod \"odh-model-controller-7998c67dc4-b5p8s\" (UID: \"4481c981-a256-41e3-a1dc-23eddfe664a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:43.787697 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:17:43.787684 2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 09 22:17:43.787769 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:17:43.787759 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4481c981-a256-41e3-a1dc-23eddfe664a4-cert podName:4481c981-a256-41e3-a1dc-23eddfe664a4 nodeName:}" failed. No retries permitted until 2026-07-09 22:17:44.28774361 +0000 UTC m=+532.385958781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4481c981-a256-41e3-a1dc-23eddfe664a4-cert") pod "odh-model-controller-7998c67dc4-b5p8s" (UID: "4481c981-a256-41e3-a1dc-23eddfe664a4") : secret "odh-model-controller-webhook-cert" not found Jul 09 22:17:43.857454 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:43.857422 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5crfx\" (UniqueName: \"kubernetes.io/projected/4481c981-a256-41e3-a1dc-23eddfe664a4-kube-api-access-5crfx\") pod \"odh-model-controller-7998c67dc4-b5p8s\" (UID: \"4481c981-a256-41e3-a1dc-23eddfe664a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:44.160069 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:44.159979 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" event={"ID":"c7605dd3-5b80-465f-912f-19b91742c54b","Type":"ContainerStarted","Data":"7c5be102665b53806fa68f963a86e1d80e7704ef674cee824a39170af3c39583"} Jul 09 22:17:44.291714 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:44.291634 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4481c981-a256-41e3-a1dc-23eddfe664a4-cert\") pod \"odh-model-controller-7998c67dc4-b5p8s\" (UID: \"4481c981-a256-41e3-a1dc-23eddfe664a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:44.294068 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:44.294045 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4481c981-a256-41e3-a1dc-23eddfe664a4-cert\") pod \"odh-model-controller-7998c67dc4-b5p8s\" (UID: \"4481c981-a256-41e3-a1dc-23eddfe664a4\") " pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:44.558741 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:44.558641 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:44.740948 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:44.740917 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b5p8s"] Jul 09 22:17:44.743610 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:17:44.743579 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4481c981_a256_41e3_a1dc_23eddfe664a4.slice/crio-52209c0ff2bb402b972326714bb13152cbe92467649b67d9d4873c3bdb0be321 WatchSource:0}: Error finding container 52209c0ff2bb402b972326714bb13152cbe92467649b67d9d4873c3bdb0be321: Status 404 returned error can't find the container with id 52209c0ff2bb402b972326714bb13152cbe92467649b67d9d4873c3bdb0be321 Jul 09 22:17:45.166568 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.166520 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" event={"ID":"4481c981-a256-41e3-a1dc-23eddfe664a4","Type":"ContainerStarted","Data":"52209c0ff2bb402b972326714bb13152cbe92467649b67d9d4873c3bdb0be321"} Jul 09 22:17:45.504170 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.504091 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-22mwk"] Jul 09 22:17:45.507509 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.507485 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:17:45.512118 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.512093 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 22:17:45.603223 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.603189 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9aa9a7d2-8530-4b4b-b0f4-35828b196c89-tls-certs\") pod \"model-serving-api-54d89bd5fc-22mwk\" (UID: \"9aa9a7d2-8530-4b4b-b0f4-35828b196c89\") " pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:17:45.603223 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.603229 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg6tc\" (UniqueName: \"kubernetes.io/projected/9aa9a7d2-8530-4b4b-b0f4-35828b196c89-kube-api-access-jg6tc\") pod \"model-serving-api-54d89bd5fc-22mwk\" (UID: \"9aa9a7d2-8530-4b4b-b0f4-35828b196c89\") " pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:17:45.670026 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.669935 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-22mwk"] Jul 09 22:17:45.704252 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.704207 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9aa9a7d2-8530-4b4b-b0f4-35828b196c89-tls-certs\") pod \"model-serving-api-54d89bd5fc-22mwk\" (UID: \"9aa9a7d2-8530-4b4b-b0f4-35828b196c89\") " pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:17:45.704252 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.704249 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg6tc\" (UniqueName: \"kubernetes.io/projected/9aa9a7d2-8530-4b4b-b0f4-35828b196c89-kube-api-access-jg6tc\") pod \"model-serving-api-54d89bd5fc-22mwk\" (UID: \"9aa9a7d2-8530-4b4b-b0f4-35828b196c89\") " pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:17:45.707277 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.707252 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9aa9a7d2-8530-4b4b-b0f4-35828b196c89-tls-certs\") pod \"model-serving-api-54d89bd5fc-22mwk\" (UID: \"9aa9a7d2-8530-4b4b-b0f4-35828b196c89\") " pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:17:45.738575 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.738549 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg6tc\" (UniqueName: \"kubernetes.io/projected/9aa9a7d2-8530-4b4b-b0f4-35828b196c89-kube-api-access-jg6tc\") pod \"model-serving-api-54d89bd5fc-22mwk\" (UID: \"9aa9a7d2-8530-4b4b-b0f4-35828b196c89\") " pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:17:45.821182 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:45.821016 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:17:46.018248 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:46.017854 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-22mwk"] Jul 09 22:17:46.019159 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:17:46.019127 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa9a7d2_8530_4b4b_b0f4_35828b196c89.slice/crio-49ff06ab62a911b629dd6ce1fdf06ed0099016c4e198cd996e26442f100f2e0d WatchSource:0}: Error finding container 49ff06ab62a911b629dd6ce1fdf06ed0099016c4e198cd996e26442f100f2e0d: Status 404 returned error can't find the container with id 49ff06ab62a911b629dd6ce1fdf06ed0099016c4e198cd996e26442f100f2e0d Jul 09 22:17:46.172605 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:46.172565 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" event={"ID":"9aa9a7d2-8530-4b4b-b0f4-35828b196c89","Type":"ContainerStarted","Data":"49ff06ab62a911b629dd6ce1fdf06ed0099016c4e198cd996e26442f100f2e0d"} Jul 09 22:17:46.962249 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:46.962223 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rmndc"] Jul 09 22:17:46.965766 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:46.965748 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:46.973910 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:46.973886 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 22:17:46.974032 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:46.973930 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-g8cdh\"" Jul 09 22:17:46.974123 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:46.974108 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 22:17:46.974183 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:46.974170 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 22:17:46.974351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:46.974307 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 22:17:47.018505 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.018469 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rmndc"] Jul 09 22:17:47.020738 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.020693 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.020902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.020812 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.020902 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.020863 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4276ca59-ea90-40b4-a82a-743936725c18-local-certs\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.021016 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.020913 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-istio-token\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.021016 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.020943 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcxsg\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-kube-api-access-fcxsg\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.021095 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.021019 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-cacerts\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.021095 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.021060 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.122265 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.122228 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.122445 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.122286 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4276ca59-ea90-40b4-a82a-743936725c18-local-certs\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.122445 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.122330 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-istio-token\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.122445 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.122357 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcxsg\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-kube-api-access-fcxsg\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.122445 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.122400 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-cacerts\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.122445 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.122428 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.122688 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.122461 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.122977 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.122937 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.125111 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.125050 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-cacerts\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.125111 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.125071 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4276ca59-ea90-40b4-a82a-743936725c18-local-certs\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.125239 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.125137 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.125574 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.125556 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.167132 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.167051 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcxsg\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-kube-api-access-fcxsg\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.169871 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.169846 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-istio-token\") pod \"istiod-openshift-gateway-7574459656-rmndc\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:47.181964 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.181805 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" event={"ID":"c7605dd3-5b80-465f-912f-19b91742c54b","Type":"ContainerStarted","Data":"09171d69fda3d238a30ad7d4fe3fb5c0154e1cf54b7eaa39fa58f87a97192716"} Jul 09 22:17:47.253739 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.253652 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-rf44x" podStartSLOduration=1.119150345 podStartE2EDuration="4.25363018s" totalImagesPullingTime="3.134479835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:17:47.252625418 +0000 UTC m=+535.350840621" watchObservedRunningTime="2026-07-09 22:17:47.25363018 +0000 UTC m=+535.351845379" Jul 09 22:17:47.290098 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:47.289858 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:48.933384 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:48.933354 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rmndc"] Jul 09 22:17:48.934626 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:17:48.934592 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4276ca59_ea90_40b4_a82a_743936725c18.slice/crio-575a10692a2cfb5a6ba1169cf7e792408ee77e121dac4514c90fd2e3ba56d6d3 WatchSource:0}: Error finding container 575a10692a2cfb5a6ba1169cf7e792408ee77e121dac4514c90fd2e3ba56d6d3: Status 404 returned error can't find the container with id 575a10692a2cfb5a6ba1169cf7e792408ee77e121dac4514c90fd2e3ba56d6d3 Jul 09 22:17:49.194338 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:49.194296 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" event={"ID":"4481c981-a256-41e3-a1dc-23eddfe664a4","Type":"ContainerStarted","Data":"f22545828e8f78e8013a1e66ec1c4ee3ee57c9e613ab2ee07635b8acfeabb924"} Jul 09 22:17:49.194534 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:49.194434 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:17:49.195556 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:49.195530 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" event={"ID":"4276ca59-ea90-40b4-a82a-743936725c18","Type":"ContainerStarted","Data":"575a10692a2cfb5a6ba1169cf7e792408ee77e121dac4514c90fd2e3ba56d6d3"} Jul 09 22:17:49.196923 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:49.196894 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" event={"ID":"9aa9a7d2-8530-4b4b-b0f4-35828b196c89","Type":"ContainerStarted","Data":"59fb70f85214fe9030cc9d0a1a6a03f4182de62797fbf6e1dae752a135838349"} Jul 09 22:17:49.197014 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:49.196941 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:17:49.238214 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:49.238157 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" podStartSLOduration=2.183540509 podStartE2EDuration="6.238141223s" totalImagesPullingTime="4.054600714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:17:49.236367407 +0000 UTC m=+537.334582612" watchObservedRunningTime="2026-07-09 22:17:49.238141223 +0000 UTC m=+537.336356414" Jul 09 22:17:49.280903 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:49.280851 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" podStartSLOduration=1.501750707 podStartE2EDuration="4.280837225s" totalImagesPullingTime="2.779086518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:17:49.279003498 +0000 UTC m=+537.377218688" watchObservedRunningTime="2026-07-09 22:17:49.280837225 +0000 UTC m=+537.379052414" Jul 09 22:17:51.235211 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:51.235174 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:17:51.235450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:51.235249 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:17:52.215603 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:52.215562 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" event={"ID":"4276ca59-ea90-40b4-a82a-743936725c18","Type":"ContainerStarted","Data":"314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0"} Jul 09 22:17:52.216686 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:52.216659 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:52.219388 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:52.219361 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-rmndc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 22:17:52.219523 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:52.219411 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" podUID="4276ca59-ea90-40b4-a82a-743936725c18" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 22:17:52.294591 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:52.294545 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" podStartSLOduration=3.996445843 podStartE2EDuration="6.29453101s" totalImagesPullingTime="2.298085167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:17:52.294316629 +0000 UTC m=+540.392531820" watchObservedRunningTime="2026-07-09 22:17:52.29453101 +0000 UTC m=+540.392746201" Jul 09 22:17:53.220597 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:53.220566 2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-rmndc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 22:17:53.220783 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:53.220613 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" podUID="4276ca59-ea90-40b4-a82a-743936725c18" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 22:17:54.224369 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:54.224337 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:17:57.403651 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.403619 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77"] Jul 09 22:17:57.408462 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.408439 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-chzl4"] Jul 09 22:17:57.408594 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.408558 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:17:57.411519 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.411499 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:17:57.416002 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.415984 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vwppc\"" Jul 09 22:17:57.416121 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.416018 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 09 22:17:57.416590 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.416570 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 09 22:17:57.416675 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.416646 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-l78vp\"" Jul 09 22:17:57.451987 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.451958 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-chzl4"] Jul 09 22:17:57.464286 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.464256 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77"] Jul 09 22:17:57.515998 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.515971 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59df5409-675c-4c7f-9f90-cd474d423f75-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xpd77\" (UID: \"59df5409-675c-4c7f-9f90-cd474d423f75\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:17:57.516168 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.516029 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpzvr\" (UniqueName: \"kubernetes.io/projected/fb70b5d3-8bde-4df1-90f4-fee384cdad1e-kube-api-access-cpzvr\") pod \"kserve-controller-manager-d6f4b65b8-chzl4\" (UID: \"fb70b5d3-8bde-4df1-90f4-fee384cdad1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:17:57.516168 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.516104 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n95zf\" (UniqueName: \"kubernetes.io/projected/59df5409-675c-4c7f-9f90-cd474d423f75-kube-api-access-n95zf\") pod \"llmisvc-controller-manager-777c6bd7d5-xpd77\" (UID: \"59df5409-675c-4c7f-9f90-cd474d423f75\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:17:57.516168 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.516145 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb70b5d3-8bde-4df1-90f4-fee384cdad1e-cert\") pod \"kserve-controller-manager-d6f4b65b8-chzl4\" (UID: \"fb70b5d3-8bde-4df1-90f4-fee384cdad1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:17:57.617324 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.617293 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n95zf\" (UniqueName: \"kubernetes.io/projected/59df5409-675c-4c7f-9f90-cd474d423f75-kube-api-access-n95zf\") pod \"llmisvc-controller-manager-777c6bd7d5-xpd77\" (UID: \"59df5409-675c-4c7f-9f90-cd474d423f75\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:17:57.617324 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.617330 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb70b5d3-8bde-4df1-90f4-fee384cdad1e-cert\") pod \"kserve-controller-manager-d6f4b65b8-chzl4\" (UID: \"fb70b5d3-8bde-4df1-90f4-fee384cdad1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:17:57.617576 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.617381 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59df5409-675c-4c7f-9f90-cd474d423f75-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xpd77\" (UID: \"59df5409-675c-4c7f-9f90-cd474d423f75\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:17:57.617576 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.617426 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpzvr\" (UniqueName: \"kubernetes.io/projected/fb70b5d3-8bde-4df1-90f4-fee384cdad1e-kube-api-access-cpzvr\") pod \"kserve-controller-manager-d6f4b65b8-chzl4\" (UID: \"fb70b5d3-8bde-4df1-90f4-fee384cdad1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:17:57.619975 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.619946 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59df5409-675c-4c7f-9f90-cd474d423f75-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xpd77\" (UID: \"59df5409-675c-4c7f-9f90-cd474d423f75\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:17:57.620091 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.619946 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb70b5d3-8bde-4df1-90f4-fee384cdad1e-cert\") pod \"kserve-controller-manager-d6f4b65b8-chzl4\" (UID: \"fb70b5d3-8bde-4df1-90f4-fee384cdad1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:17:57.638467 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.638440 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpzvr\" (UniqueName: \"kubernetes.io/projected/fb70b5d3-8bde-4df1-90f4-fee384cdad1e-kube-api-access-cpzvr\") pod \"kserve-controller-manager-d6f4b65b8-chzl4\" (UID: \"fb70b5d3-8bde-4df1-90f4-fee384cdad1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:17:57.666143 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.666083 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n95zf\" (UniqueName: \"kubernetes.io/projected/59df5409-675c-4c7f-9f90-cd474d423f75-kube-api-access-n95zf\") pod \"llmisvc-controller-manager-777c6bd7d5-xpd77\" (UID: \"59df5409-675c-4c7f-9f90-cd474d423f75\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:17:57.720048 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.720017 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:17:57.725330 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.725301 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:17:57.886225 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.886201 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77"] Jul 09 22:17:57.887159 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:17:57.887131 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod59df5409_675c_4c7f_9f90_cd474d423f75.slice/crio-a85dbb6b745e3915446fde55788182626c4583c8a18739a58c10e8e96c0f90ec WatchSource:0}: Error finding container a85dbb6b745e3915446fde55788182626c4583c8a18739a58c10e8e96c0f90ec: Status 404 returned error can't find the container with id a85dbb6b745e3915446fde55788182626c4583c8a18739a58c10e8e96c0f90ec Jul 09 22:17:57.919362 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:57.919335 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-chzl4"] Jul 09 22:17:57.920870 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:17:57.920844 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb70b5d3_8bde_4df1_90f4_fee384cdad1e.slice/crio-2d06b4536a18749f425e1c4edd53e88617e338d07ddc744d475b0beb393b99f2 WatchSource:0}: Error finding container 2d06b4536a18749f425e1c4edd53e88617e338d07ddc744d475b0beb393b99f2: Status 404 returned error can't find the container with id 2d06b4536a18749f425e1c4edd53e88617e338d07ddc744d475b0beb393b99f2 Jul 09 22:17:58.244436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:58.244347 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" event={"ID":"59df5409-675c-4c7f-9f90-cd474d423f75","Type":"ContainerStarted","Data":"a85dbb6b745e3915446fde55788182626c4583c8a18739a58c10e8e96c0f90ec"} Jul 09 22:17:58.245506 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:17:58.245478 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" event={"ID":"fb70b5d3-8bde-4df1-90f4-fee384cdad1e","Type":"ContainerStarted","Data":"2d06b4536a18749f425e1c4edd53e88617e338d07ddc744d475b0beb393b99f2"} Jul 09 22:18:00.204513 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:00.204487 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-b5p8s" Jul 09 22:18:00.206913 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:00.206891 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-22mwk" Jul 09 22:18:01.006236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.006201 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rmndc"] Jul 09 22:18:01.006424 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.006395 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" podUID="4276ca59-ea90-40b4-a82a-743936725c18" containerName="discovery" containerID="cri-o://314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0" gracePeriod=30 Jul 09 22:18:01.496039 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.496019 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:18:01.657731 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.657643 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-istio-token\") pod \"4276ca59-ea90-40b4-a82a-743936725c18\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " Jul 09 22:18:01.657731 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.657707 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-ca-configmap\") pod \"4276ca59-ea90-40b4-a82a-743936725c18\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " Jul 09 22:18:01.657911 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.657754 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-dns-cert\") pod \"4276ca59-ea90-40b4-a82a-743936725c18\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " Jul 09 22:18:01.657911 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.657773 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4276ca59-ea90-40b4-a82a-743936725c18-local-certs\") pod \"4276ca59-ea90-40b4-a82a-743936725c18\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " Jul 09 22:18:01.657911 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.657800 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-cacerts\") pod \"4276ca59-ea90-40b4-a82a-743936725c18\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " Jul 09 22:18:01.657911 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.657874 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-kubeconfig\") pod \"4276ca59-ea90-40b4-a82a-743936725c18\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " Jul 09 22:18:01.657911 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.657901 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcxsg\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-kube-api-access-fcxsg\") pod \"4276ca59-ea90-40b4-a82a-743936725c18\" (UID: \"4276ca59-ea90-40b4-a82a-743936725c18\") " Jul 09 22:18:01.658251 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.658215 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "4276ca59-ea90-40b4-a82a-743936725c18" (UID: "4276ca59-ea90-40b4-a82a-743936725c18"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:18:01.658591 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.658560 2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-ca-configmap\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:18:01.660127 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.660098 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-cacerts" (OuterVolumeSpecName: "cacerts") pod "4276ca59-ea90-40b4-a82a-743936725c18" (UID: "4276ca59-ea90-40b4-a82a-743936725c18"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:18:01.660226 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.660134 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "4276ca59-ea90-40b4-a82a-743936725c18" (UID: "4276ca59-ea90-40b4-a82a-743936725c18"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:18:01.660279 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.660222 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-istio-token" (OuterVolumeSpecName: "istio-token") pod "4276ca59-ea90-40b4-a82a-743936725c18" (UID: "4276ca59-ea90-40b4-a82a-743936725c18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:18:01.660279 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.660226 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4276ca59-ea90-40b4-a82a-743936725c18-local-certs" (OuterVolumeSpecName: "local-certs") pod "4276ca59-ea90-40b4-a82a-743936725c18" (UID: "4276ca59-ea90-40b4-a82a-743936725c18"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:18:01.660429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.660410 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-kube-api-access-fcxsg" (OuterVolumeSpecName: "kube-api-access-fcxsg") pod "4276ca59-ea90-40b4-a82a-743936725c18" (UID: "4276ca59-ea90-40b4-a82a-743936725c18"). InnerVolumeSpecName "kube-api-access-fcxsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:18:01.660520 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.660498 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "4276ca59-ea90-40b4-a82a-743936725c18" (UID: "4276ca59-ea90-40b4-a82a-743936725c18"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 22:18:01.759501 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.759469 2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4276ca59-ea90-40b4-a82a-743936725c18-local-certs\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:18:01.759501 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.759500 2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-cacerts\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:18:01.759710 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.759513 2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-kubeconfig\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:18:01.759710 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.759526 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcxsg\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-kube-api-access-fcxsg\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:18:01.759710 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.759539 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4276ca59-ea90-40b4-a82a-743936725c18-istio-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:18:01.759710 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:01.759549 2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4276ca59-ea90-40b4-a82a-743936725c18-istio-csr-dns-cert\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:18:02.268928 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.268890 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" event={"ID":"fb70b5d3-8bde-4df1-90f4-fee384cdad1e","Type":"ContainerStarted","Data":"93f1ded2789afda5bbcde5560ee5b74f18144f384f0d2b6bf0d797a337030ac0"} Jul 09 22:18:02.269143 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.268989 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:18:02.270396 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.270368 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" event={"ID":"59df5409-675c-4c7f-9f90-cd474d423f75","Type":"ContainerStarted","Data":"985fbf127eaa49d91adc38051e9b12ad636e822a8c0ecdb8df81a7269f695086"} Jul 09 22:18:02.270501 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.270447 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:18:02.271615 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.271596 2576 generic.go:356] "Generic (PLEG): container finished" podID="4276ca59-ea90-40b4-a82a-743936725c18" containerID="314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0" exitCode=0 Jul 09 22:18:02.271686 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.271622 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" event={"ID":"4276ca59-ea90-40b4-a82a-743936725c18","Type":"ContainerDied","Data":"314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0"} Jul 09 22:18:02.271686 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.271640 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" Jul 09 22:18:02.271686 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.271649 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rmndc" event={"ID":"4276ca59-ea90-40b4-a82a-743936725c18","Type":"ContainerDied","Data":"575a10692a2cfb5a6ba1169cf7e792408ee77e121dac4514c90fd2e3ba56d6d3"} Jul 09 22:18:02.271686 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.271669 2576 scope.go:117] "RemoveContainer" containerID="314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0" Jul 09 22:18:02.280793 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.280777 2576 scope.go:117] "RemoveContainer" containerID="314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0" Jul 09 22:18:02.281026 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:18:02.281007 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0\": container with ID starting with 314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0 not found: ID does not exist" containerID="314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0" Jul 09 22:18:02.281091 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.281034 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0"} err="failed to get container status \"314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0\": rpc error: code = NotFound desc = could not find container \"314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0\": container with ID starting with 314c38439ee6013cf9a041561bf2d09791991d4d64011942a63d7c31c2393ff0 not found: ID does not exist" Jul 09 22:18:02.304384 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.304339 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" podStartSLOduration=1.862547279 podStartE2EDuration="5.304327081s" totalImagesPullingTime="3.441779802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:18:02.303452788 +0000 UTC m=+550.401667977" watchObservedRunningTime="2026-07-09 22:18:02.304327081 +0000 UTC m=+550.402542318" Jul 09 22:18:02.347736 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.347695 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rmndc"] Jul 09 22:18:02.358895 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.358869 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rmndc"] Jul 09 22:18:02.388194 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.388165 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4276ca59-ea90-40b4-a82a-743936725c18" path="/var/lib/kubelet/pods/4276ca59-ea90-40b4-a82a-743936725c18/volumes" Jul 09 22:18:02.412221 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:02.412174 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" podStartSLOduration=1.9335592959999999 podStartE2EDuration="5.412156909s" totalImagesPullingTime="3.478597613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:18:02.411123845 +0000 UTC m=+550.509339035" watchObservedRunningTime="2026-07-09 22:18:02.412156909 +0000 UTC m=+550.510372098" Jul 09 22:18:33.279670 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:33.279640 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xpd77" Jul 09 22:18:33.282680 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:33.282658 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chzl4" Jul 09 22:18:44.057254 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.057220 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ssn9n"] Jul 09 22:18:44.057626 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.057537 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4276ca59-ea90-40b4-a82a-743936725c18" containerName="discovery" Jul 09 22:18:44.057626 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.057549 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4276ca59-ea90-40b4-a82a-743936725c18" containerName="discovery" Jul 09 22:18:44.057626 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.057619 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4276ca59-ea90-40b4-a82a-743936725c18" containerName="discovery" Jul 09 22:18:44.068270 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.068245 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" Jul 09 22:18:44.072798 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.072775 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 22:18:44.074952 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.074924 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-vp4j6\"" Jul 09 22:18:44.075055 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.075005 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 22:18:44.083340 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.083315 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ssn9n"] Jul 09 22:18:44.215081 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.215045 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m8bb\" (UniqueName: \"kubernetes.io/projected/51e24f67-5915-4051-a7d7-add4395bb687-kube-api-access-8m8bb\") pod \"kuadrant-operator-catalog-ssn9n\" (UID: \"51e24f67-5915-4051-a7d7-add4395bb687\") " pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" Jul 09 22:18:44.315730 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.315639 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m8bb\" (UniqueName: \"kubernetes.io/projected/51e24f67-5915-4051-a7d7-add4395bb687-kube-api-access-8m8bb\") pod \"kuadrant-operator-catalog-ssn9n\" (UID: \"51e24f67-5915-4051-a7d7-add4395bb687\") " pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" Jul 09 22:18:44.325305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.325276 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m8bb\" (UniqueName: \"kubernetes.io/projected/51e24f67-5915-4051-a7d7-add4395bb687-kube-api-access-8m8bb\") pod \"kuadrant-operator-catalog-ssn9n\" (UID: \"51e24f67-5915-4051-a7d7-add4395bb687\") " pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" Jul 09 22:18:44.377536 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.377502 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" Jul 09 22:18:44.388409 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.388223 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ssn9n"] Jul 09 22:18:44.518638 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.518614 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ssn9n"] Jul 09 22:18:44.520248 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:18:44.520222 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e24f67_5915_4051_a7d7_add4395bb687.slice/crio-48f7fbc88e7f3985f00f427d1239c81aee5494e466782c39d207ef9ad5fbcdef WatchSource:0}: Error finding container 48f7fbc88e7f3985f00f427d1239c81aee5494e466782c39d207ef9ad5fbcdef: Status 404 returned error can't find the container with id 48f7fbc88e7f3985f00f427d1239c81aee5494e466782c39d207ef9ad5fbcdef Jul 09 22:18:44.593978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.593943 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mfbnm"] Jul 09 22:18:44.598522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.598503 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" Jul 09 22:18:44.605214 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.605185 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mfbnm"] Jul 09 22:18:44.718436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.718407 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpbzt\" (UniqueName: \"kubernetes.io/projected/b5a6d0b1-5192-41b6-94c4-abcd18e90091-kube-api-access-cpbzt\") pod \"kuadrant-operator-catalog-mfbnm\" (UID: \"b5a6d0b1-5192-41b6-94c4-abcd18e90091\") " pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" Jul 09 22:18:44.819655 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.819614 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpbzt\" (UniqueName: \"kubernetes.io/projected/b5a6d0b1-5192-41b6-94c4-abcd18e90091-kube-api-access-cpbzt\") pod \"kuadrant-operator-catalog-mfbnm\" (UID: \"b5a6d0b1-5192-41b6-94c4-abcd18e90091\") " pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" Jul 09 22:18:44.829859 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.829825 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpbzt\" (UniqueName: \"kubernetes.io/projected/b5a6d0b1-5192-41b6-94c4-abcd18e90091-kube-api-access-cpbzt\") pod \"kuadrant-operator-catalog-mfbnm\" (UID: \"b5a6d0b1-5192-41b6-94c4-abcd18e90091\") " pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" Jul 09 22:18:44.908921 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:44.908836 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" Jul 09 22:18:45.036688 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:45.036666 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mfbnm"] Jul 09 22:18:45.038546 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:18:45.038506 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a6d0b1_5192_41b6_94c4_abcd18e90091.slice/crio-d1108ed2e27eb44844b45dece7ba81c46071fe9f35db6c1ecd8ca61e6f96944a WatchSource:0}: Error finding container d1108ed2e27eb44844b45dece7ba81c46071fe9f35db6c1ecd8ca61e6f96944a: Status 404 returned error can't find the container with id d1108ed2e27eb44844b45dece7ba81c46071fe9f35db6c1ecd8ca61e6f96944a Jul 09 22:18:45.468386 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:45.468347 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" event={"ID":"b5a6d0b1-5192-41b6-94c4-abcd18e90091","Type":"ContainerStarted","Data":"d1108ed2e27eb44844b45dece7ba81c46071fe9f35db6c1ecd8ca61e6f96944a"} Jul 09 22:18:45.469557 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:45.469533 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" event={"ID":"51e24f67-5915-4051-a7d7-add4395bb687","Type":"ContainerStarted","Data":"48f7fbc88e7f3985f00f427d1239c81aee5494e466782c39d207ef9ad5fbcdef"} Jul 09 22:18:47.481490 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:47.481388 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" event={"ID":"b5a6d0b1-5192-41b6-94c4-abcd18e90091","Type":"ContainerStarted","Data":"37810bf1135737f692b23547786f125f7e938659c02d9676e20d2b99650c7652"} Jul 09 22:18:47.482930 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:47.482906 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" event={"ID":"51e24f67-5915-4051-a7d7-add4395bb687","Type":"ContainerStarted","Data":"873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661"} Jul 09 22:18:47.483040 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:47.482999 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" podUID="51e24f67-5915-4051-a7d7-add4395bb687" containerName="registry-server" containerID="cri-o://873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661" gracePeriod=2 Jul 09 22:18:47.501974 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:47.501921 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" podStartSLOduration=1.356270283 podStartE2EDuration="3.501905302s" totalImagesPullingTime="2.145635019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:18:47.500522545 +0000 UTC m=+595.598737736" watchObservedRunningTime="2026-07-09 22:18:47.501905302 +0000 UTC m=+595.600120469" Jul 09 22:18:47.520556 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:47.520497 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" podStartSLOduration=0.859782648 podStartE2EDuration="3.52047858s" totalImagesPullingTime="2.660695932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:18:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:18:47.519279054 +0000 UTC m=+595.617494266" watchObservedRunningTime="2026-07-09 22:18:47.52047858 +0000 UTC m=+595.618693771" Jul 09 22:18:47.730154 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:47.730129 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" Jul 09 22:18:47.848271 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:47.848243 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m8bb\" (UniqueName: \"kubernetes.io/projected/51e24f67-5915-4051-a7d7-add4395bb687-kube-api-access-8m8bb\") pod \"51e24f67-5915-4051-a7d7-add4395bb687\" (UID: \"51e24f67-5915-4051-a7d7-add4395bb687\") " Jul 09 22:18:47.850367 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:47.850336 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e24f67-5915-4051-a7d7-add4395bb687-kube-api-access-8m8bb" (OuterVolumeSpecName: "kube-api-access-8m8bb") pod "51e24f67-5915-4051-a7d7-add4395bb687" (UID: "51e24f67-5915-4051-a7d7-add4395bb687"). InnerVolumeSpecName "kube-api-access-8m8bb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:18:47.949632 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:47.949595 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m8bb\" (UniqueName: \"kubernetes.io/projected/51e24f67-5915-4051-a7d7-add4395bb687-kube-api-access-8m8bb\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:18:48.488597 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:48.488509 2576 generic.go:356] "Generic (PLEG): container finished" podID="51e24f67-5915-4051-a7d7-add4395bb687" containerID="873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661" exitCode=0 Jul 09 22:18:48.488597 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:48.488559 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" Jul 09 22:18:48.489147 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:48.488595 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" event={"ID":"51e24f67-5915-4051-a7d7-add4395bb687","Type":"ContainerDied","Data":"873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661"} Jul 09 22:18:48.489147 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:48.488644 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ssn9n" event={"ID":"51e24f67-5915-4051-a7d7-add4395bb687","Type":"ContainerDied","Data":"48f7fbc88e7f3985f00f427d1239c81aee5494e466782c39d207ef9ad5fbcdef"} Jul 09 22:18:48.489147 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:48.488669 2576 scope.go:117] "RemoveContainer" containerID="873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661" Jul 09 22:18:48.497066 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:48.497048 2576 scope.go:117] "RemoveContainer" containerID="873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661" Jul 09 22:18:48.497333 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:18:48.497313 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661\": container with ID starting with 873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661 not found: ID does not exist" containerID="873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661" Jul 09 22:18:48.497378 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:48.497341 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661"} err="failed to get container status \"873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661\": rpc error: code = NotFound desc = could not find container \"873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661\": container with ID starting with 873c5cbb0bae1bc74aaad75ae9476824b98430a90bc54120cc47c2eb26e2f661 not found: ID does not exist" Jul 09 22:18:48.511358 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:48.511327 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ssn9n"] Jul 09 22:18:48.518228 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:48.518205 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ssn9n"] Jul 09 22:18:50.387661 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:50.387631 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51e24f67-5915-4051-a7d7-add4395bb687" path="/var/lib/kubelet/pods/51e24f67-5915-4051-a7d7-add4395bb687/volumes" Jul 09 22:18:52.799566 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:52.799538 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:18:52.800086 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:52.800067 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:18:54.909218 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:54.909176 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" Jul 09 22:18:54.909654 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:54.909242 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" Jul 09 22:18:54.932265 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:54.932240 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" Jul 09 22:18:55.542672 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:18:55.542644 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-mfbnm" Jul 09 22:19:16.035837 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.035761 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp"] Jul 09 22:19:16.036280 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.036157 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e24f67-5915-4051-a7d7-add4395bb687" containerName="registry-server" Jul 09 22:19:16.036280 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.036169 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e24f67-5915-4051-a7d7-add4395bb687" containerName="registry-server" Jul 09 22:19:16.036280 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.036230 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e24f67-5915-4051-a7d7-add4395bb687" containerName="registry-server" Jul 09 22:19:16.039235 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.039217 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" Jul 09 22:19:16.044682 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.044655 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 09 22:19:16.044847 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.044746 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-6vd8r\"" Jul 09 22:19:16.053906 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.053883 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp"] Jul 09 22:19:16.079214 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.079191 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jkbj\" (UniqueName: \"kubernetes.io/projected/e4a4d0b2-e041-45c1-8505-e3d08827d853-kube-api-access-9jkbj\") pod \"dns-operator-controller-manager-864dd78c4c-h8kcp\" (UID: \"e4a4d0b2-e041-45c1-8505-e3d08827d853\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" Jul 09 22:19:16.180414 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.180380 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jkbj\" (UniqueName: \"kubernetes.io/projected/e4a4d0b2-e041-45c1-8505-e3d08827d853-kube-api-access-9jkbj\") pod \"dns-operator-controller-manager-864dd78c4c-h8kcp\" (UID: \"e4a4d0b2-e041-45c1-8505-e3d08827d853\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" Jul 09 22:19:16.210704 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.210671 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jkbj\" (UniqueName: \"kubernetes.io/projected/e4a4d0b2-e041-45c1-8505-e3d08827d853-kube-api-access-9jkbj\") pod \"dns-operator-controller-manager-864dd78c4c-h8kcp\" (UID: \"e4a4d0b2-e041-45c1-8505-e3d08827d853\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" Jul 09 22:19:16.349372 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.349340 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" Jul 09 22:19:16.476700 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.476674 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp"] Jul 09 22:19:16.477924 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:19:16.477897 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4a4d0b2_e041_45c1_8505_e3d08827d853.slice/crio-06e1eac417984a644d6e382ff34213284b14be34d077176f6aa79b4ac19bd1ac WatchSource:0}: Error finding container 06e1eac417984a644d6e382ff34213284b14be34d077176f6aa79b4ac19bd1ac: Status 404 returned error can't find the container with id 06e1eac417984a644d6e382ff34213284b14be34d077176f6aa79b4ac19bd1ac Jul 09 22:19:16.616674 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:16.616585 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" event={"ID":"e4a4d0b2-e041-45c1-8505-e3d08827d853","Type":"ContainerStarted","Data":"06e1eac417984a644d6e382ff34213284b14be34d077176f6aa79b4ac19bd1ac"} Jul 09 22:19:19.633932 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:19.633895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" event={"ID":"e4a4d0b2-e041-45c1-8505-e3d08827d853","Type":"ContainerStarted","Data":"cb747f216d6150499762653230ce9a5dc0513d1ec0fd95ff1fac2256a58605d4"} Jul 09 22:19:19.634313 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:19.634004 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" Jul 09 22:19:19.672410 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:19.672359 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" podStartSLOduration=1.468302861 podStartE2EDuration="3.672346375s" totalImagesPullingTime="2.204043514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:19.668938223 +0000 UTC m=+627.767153412" watchObservedRunningTime="2026-07-09 22:19:19.672346375 +0000 UTC m=+627.770561564" Jul 09 22:19:21.395951 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.395917 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87"] Jul 09 22:19:21.404581 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.404556 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:21.408074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.408053 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j5cdt\"" Jul 09 22:19:21.412458 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.412434 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87"] Jul 09 22:19:21.426389 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.426369 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfptq\" (UniqueName: \"kubernetes.io/projected/2a646bbe-909c-4174-83a7-84624c2296bf-kube-api-access-dfptq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" (UID: \"2a646bbe-909c-4174-83a7-84624c2296bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:21.426491 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.426402 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a646bbe-909c-4174-83a7-84624c2296bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" (UID: \"2a646bbe-909c-4174-83a7-84624c2296bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:21.527537 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.527509 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfptq\" (UniqueName: \"kubernetes.io/projected/2a646bbe-909c-4174-83a7-84624c2296bf-kube-api-access-dfptq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" (UID: \"2a646bbe-909c-4174-83a7-84624c2296bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:21.527691 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.527548 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a646bbe-909c-4174-83a7-84624c2296bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" (UID: \"2a646bbe-909c-4174-83a7-84624c2296bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:21.527926 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.527911 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a646bbe-909c-4174-83a7-84624c2296bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" (UID: \"2a646bbe-909c-4174-83a7-84624c2296bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:21.537077 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.537057 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfptq\" (UniqueName: \"kubernetes.io/projected/2a646bbe-909c-4174-83a7-84624c2296bf-kube-api-access-dfptq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" (UID: \"2a646bbe-909c-4174-83a7-84624c2296bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:21.715499 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.715418 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:21.857473 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:19:21.857439 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a646bbe_909c_4174_83a7_84624c2296bf.slice/crio-b8461f3ab19647bdfa39a0ea21e8b81e1f330f2ee929dd147d57e08303e7c828 WatchSource:0}: Error finding container b8461f3ab19647bdfa39a0ea21e8b81e1f330f2ee929dd147d57e08303e7c828: Status 404 returned error can't find the container with id b8461f3ab19647bdfa39a0ea21e8b81e1f330f2ee929dd147d57e08303e7c828 Jul 09 22:19:21.861167 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:21.861144 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87"] Jul 09 22:19:22.648385 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:22.648348 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" event={"ID":"2a646bbe-909c-4174-83a7-84624c2296bf","Type":"ContainerStarted","Data":"b8461f3ab19647bdfa39a0ea21e8b81e1f330f2ee929dd147d57e08303e7c828"} Jul 09 22:19:25.111229 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.111195 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz"] Jul 09 22:19:25.114652 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.114636 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" Jul 09 22:19:25.121412 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.121394 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lrw99\"" Jul 09 22:19:25.130797 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.130771 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz"] Jul 09 22:19:25.160130 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.160096 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjblj\" (UniqueName: \"kubernetes.io/projected/1fcbc8f6-1604-4d51-81c0-1b6111e23358-kube-api-access-wjblj\") pod \"authorino-operator-75cc8d5cb7-5jtdz\" (UID: \"1fcbc8f6-1604-4d51-81c0-1b6111e23358\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" Jul 09 22:19:25.260697 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.260666 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjblj\" (UniqueName: \"kubernetes.io/projected/1fcbc8f6-1604-4d51-81c0-1b6111e23358-kube-api-access-wjblj\") pod \"authorino-operator-75cc8d5cb7-5jtdz\" (UID: \"1fcbc8f6-1604-4d51-81c0-1b6111e23358\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" Jul 09 22:19:25.273179 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.273159 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjblj\" (UniqueName: \"kubernetes.io/projected/1fcbc8f6-1604-4d51-81c0-1b6111e23358-kube-api-access-wjblj\") pod \"authorino-operator-75cc8d5cb7-5jtdz\" (UID: \"1fcbc8f6-1604-4d51-81c0-1b6111e23358\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" Jul 09 22:19:25.425072 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.424985 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" Jul 09 22:19:25.584201 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.584179 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz"] Jul 09 22:19:25.585798 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:19:25.585767 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fcbc8f6_1604_4d51_81c0_1b6111e23358.slice/crio-7adeb3fdc713c9e429ebc0876f44cdf5af6e6101b55e50d254192990a88ad57d WatchSource:0}: Error finding container 7adeb3fdc713c9e429ebc0876f44cdf5af6e6101b55e50d254192990a88ad57d: Status 404 returned error can't find the container with id 7adeb3fdc713c9e429ebc0876f44cdf5af6e6101b55e50d254192990a88ad57d Jul 09 22:19:25.662248 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:25.662211 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" event={"ID":"1fcbc8f6-1604-4d51-81c0-1b6111e23358","Type":"ContainerStarted","Data":"7adeb3fdc713c9e429ebc0876f44cdf5af6e6101b55e50d254192990a88ad57d"} Jul 09 22:19:28.231922 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:28.231890 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp"] Jul 09 22:19:28.235565 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:28.235544 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:28.239844 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:28.239821 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6cp95\"" Jul 09 22:19:28.249477 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:28.249459 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp"] Jul 09 22:19:28.294676 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:28.294635 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9lf8\" (UniqueName: \"kubernetes.io/projected/ddd9366c-a137-4080-9f1d-5a680bee2376-kube-api-access-d9lf8\") pod \"limitador-operator-controller-manager-859d7b78bd-999mp\" (UID: \"ddd9366c-a137-4080-9f1d-5a680bee2376\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:28.396026 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:28.395993 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9lf8\" (UniqueName: \"kubernetes.io/projected/ddd9366c-a137-4080-9f1d-5a680bee2376-kube-api-access-d9lf8\") pod \"limitador-operator-controller-manager-859d7b78bd-999mp\" (UID: \"ddd9366c-a137-4080-9f1d-5a680bee2376\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:28.409025 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:28.408997 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9lf8\" (UniqueName: \"kubernetes.io/projected/ddd9366c-a137-4080-9f1d-5a680bee2376-kube-api-access-d9lf8\") pod \"limitador-operator-controller-manager-859d7b78bd-999mp\" (UID: \"ddd9366c-a137-4080-9f1d-5a680bee2376\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:28.548916 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:28.548830 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:29.227522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:29.227498 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp"] Jul 09 22:19:29.229841 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:19:29.229815 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd9366c_a137_4080_9f1d_5a680bee2376.slice/crio-15ca04a867f310cb27caf9bba56b45446144fff358169084fb6e34906b3201ee WatchSource:0}: Error finding container 15ca04a867f310cb27caf9bba56b45446144fff358169084fb6e34906b3201ee: Status 404 returned error can't find the container with id 15ca04a867f310cb27caf9bba56b45446144fff358169084fb6e34906b3201ee Jul 09 22:19:29.684369 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:29.684336 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" event={"ID":"2a646bbe-909c-4174-83a7-84624c2296bf","Type":"ContainerStarted","Data":"d0fa0d772dd3c161d6032519a9376f8a74e5b9a82f640269c538ea9452194b9d"} Jul 09 22:19:29.684865 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:29.684459 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:29.685603 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:29.685580 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" event={"ID":"ddd9366c-a137-4080-9f1d-5a680bee2376","Type":"ContainerStarted","Data":"15ca04a867f310cb27caf9bba56b45446144fff358169084fb6e34906b3201ee"} Jul 09 22:19:29.686938 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:29.686913 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" event={"ID":"1fcbc8f6-1604-4d51-81c0-1b6111e23358","Type":"ContainerStarted","Data":"cfd73cdedf83f1d81a453f64974f346f79ed0759cf5b1349b95505acb2dc9c28"} Jul 09 22:19:29.687040 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:29.687026 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" Jul 09 22:19:29.710025 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:29.709979 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" podStartSLOduration=1.468353289 podStartE2EDuration="8.709956693s" totalImagesPullingTime="7.241603404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:29.706729855 +0000 UTC m=+637.804945044" watchObservedRunningTime="2026-07-09 22:19:29.709956693 +0000 UTC m=+637.808171882" Jul 09 22:19:29.736058 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:29.736009 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" podStartSLOduration=1.248748953 podStartE2EDuration="4.735992418s" totalImagesPullingTime="3.487243465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:29.735967523 +0000 UTC m=+637.834182716" watchObservedRunningTime="2026-07-09 22:19:29.735992418 +0000 UTC m=+637.834207607" Jul 09 22:19:30.641166 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:30.641132 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h8kcp" Jul 09 22:19:31.698369 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:31.698337 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" event={"ID":"ddd9366c-a137-4080-9f1d-5a680bee2376","Type":"ContainerStarted","Data":"ba01bb57fdef1e44d9753ad2d86cbb7527711a83ca4c7d032c3cfda2ed1ebfa6"} Jul 09 22:19:31.698769 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:31.698431 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:31.723480 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:31.723426 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" podStartSLOduration=1.690297844 podStartE2EDuration="3.723411863s" totalImagesPullingTime="2.033114019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:31.722214367 +0000 UTC m=+639.820429570" watchObservedRunningTime="2026-07-09 22:19:31.723411863 +0000 UTC m=+639.821627108" Jul 09 22:19:40.694308 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:40.694274 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:40.694685 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:40.694482 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5jtdz" Jul 09 22:19:42.520780 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.520746 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87"] Jul 09 22:19:42.521187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.520936 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" containerName="manager" containerID="cri-o://d0fa0d772dd3c161d6032519a9376f8a74e5b9a82f640269c538ea9452194b9d" gracePeriod=2 Jul 09 22:19:42.524612 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.524573 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87"] Jul 09 22:19:42.524947 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.524921 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:42.539232 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.539201 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp"] Jul 09 22:19:42.539469 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.539440 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" containerName="manager" containerID="cri-o://ba01bb57fdef1e44d9753ad2d86cbb7527711a83ca4c7d032c3cfda2ed1ebfa6" gracePeriod=2 Jul 09 22:19:42.541761 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.541582 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:42.542902 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.542679 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:42.548616 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.548300 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd"] Jul 09 22:19:42.548855 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.548827 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" containerName="manager" Jul 09 22:19:42.548855 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.548849 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" containerName="manager" Jul 09 22:19:42.548978 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.548957 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" containerName="manager" Jul 09 22:19:42.552101 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.552079 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp"] Jul 09 22:19:42.552210 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.552183 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:42.555732 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.555687 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:42.558785 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.558757 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-999mp\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:42.567947 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.567921 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k"] Jul 09 22:19:42.568297 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.568284 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" containerName="manager" Jul 09 22:19:42.568361 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.568298 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" containerName="manager" Jul 09 22:19:42.568411 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.568367 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" containerName="manager" Jul 09 22:19:42.571276 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.571258 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd"] Jul 09 22:19:42.571378 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.571338 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" Jul 09 22:19:42.584236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.584209 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k"] Jul 09 22:19:42.588697 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.588661 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:42.615774 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.615742 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-999mp\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:42.623120 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.623087 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p76l\" (UniqueName: \"kubernetes.io/projected/acdaa1d4-cc46-40ab-8192-c8128265ed97-kube-api-access-2p76l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8knd\" (UID: \"acdaa1d4-cc46-40ab-8192-c8128265ed97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:42.623190 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.623135 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acdaa1d4-cc46-40ab-8192-c8128265ed97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8knd\" (UID: \"acdaa1d4-cc46-40ab-8192-c8128265ed97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:42.701399 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.701368 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs"] Jul 09 22:19:42.704874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.704851 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:42.723841 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.723812 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p76l\" (UniqueName: \"kubernetes.io/projected/acdaa1d4-cc46-40ab-8192-c8128265ed97-kube-api-access-2p76l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8knd\" (UID: \"acdaa1d4-cc46-40ab-8192-c8128265ed97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:42.723975 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.723863 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acdaa1d4-cc46-40ab-8192-c8128265ed97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8knd\" (UID: \"acdaa1d4-cc46-40ab-8192-c8128265ed97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:42.723975 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.723897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmt7n\" (UniqueName: \"kubernetes.io/projected/b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5-kube-api-access-zmt7n\") pod \"limitador-operator-controller-manager-859d7b78bd-k456k\" (UID: \"b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" Jul 09 22:19:42.724777 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.724750 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acdaa1d4-cc46-40ab-8192-c8128265ed97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8knd\" (UID: \"acdaa1d4-cc46-40ab-8192-c8128265ed97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:42.731879 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.731849 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs"] Jul 09 22:19:42.735843 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.735805 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p76l\" (UniqueName: \"kubernetes.io/projected/acdaa1d4-cc46-40ab-8192-c8128265ed97-kube-api-access-2p76l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w8knd\" (UID: \"acdaa1d4-cc46-40ab-8192-c8128265ed97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:42.753313 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.753281 2576 generic.go:356] "Generic (PLEG): container finished" podID="2a646bbe-909c-4174-83a7-84624c2296bf" containerID="d0fa0d772dd3c161d6032519a9376f8a74e5b9a82f640269c538ea9452194b9d" exitCode=0 Jul 09 22:19:42.755231 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.755203 2576 generic.go:356] "Generic (PLEG): container finished" podID="ddd9366c-a137-4080-9f1d-5a680bee2376" containerID="ba01bb57fdef1e44d9753ad2d86cbb7527711a83ca4c7d032c3cfda2ed1ebfa6" exitCode=0 Jul 09 22:19:42.771728 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.771653 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:42.774976 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.774947 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-999mp\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:42.788661 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.788630 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:42.791707 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.791688 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:42.792099 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.792078 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-999mp\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:42.795537 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.795515 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:42.799126 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.799105 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-999mp\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:42.802210 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:42.802190 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:42.825263 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.825232 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e9a2d1-1784-46f0-85e6-bd003c923d18-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6zgrs\" (UID: \"33e9a2d1-1784-46f0-85e6-bd003c923d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:42.825368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.825280 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4hbk\" (UniqueName: \"kubernetes.io/projected/33e9a2d1-1784-46f0-85e6-bd003c923d18-kube-api-access-b4hbk\") pod \"kuadrant-operator-controller-manager-86878c769f-6zgrs\" (UID: \"33e9a2d1-1784-46f0-85e6-bd003c923d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:42.825368 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.825333 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmt7n\" (UniqueName: \"kubernetes.io/projected/b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5-kube-api-access-zmt7n\") pod \"limitador-operator-controller-manager-859d7b78bd-k456k\" (UID: \"b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" Jul 09 22:19:42.839860 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.839835 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmt7n\" (UniqueName: \"kubernetes.io/projected/b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5-kube-api-access-zmt7n\") pod \"limitador-operator-controller-manager-859d7b78bd-k456k\" (UID: \"b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" Jul 09 22:19:42.926229 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.926204 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a646bbe-909c-4174-83a7-84624c2296bf-extensions-socket-volume\") pod \"2a646bbe-909c-4174-83a7-84624c2296bf\" (UID: \"2a646bbe-909c-4174-83a7-84624c2296bf\") " Jul 09 22:19:42.926400 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.926237 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfptq\" (UniqueName: \"kubernetes.io/projected/2a646bbe-909c-4174-83a7-84624c2296bf-kube-api-access-dfptq\") pod \"2a646bbe-909c-4174-83a7-84624c2296bf\" (UID: \"2a646bbe-909c-4174-83a7-84624c2296bf\") " Jul 09 22:19:42.926400 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.926313 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9lf8\" (UniqueName: \"kubernetes.io/projected/ddd9366c-a137-4080-9f1d-5a680bee2376-kube-api-access-d9lf8\") pod \"ddd9366c-a137-4080-9f1d-5a680bee2376\" (UID: \"ddd9366c-a137-4080-9f1d-5a680bee2376\") " Jul 09 22:19:42.926516 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.926435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e9a2d1-1784-46f0-85e6-bd003c923d18-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6zgrs\" (UID: \"33e9a2d1-1784-46f0-85e6-bd003c923d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:42.926516 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.926458 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4hbk\" (UniqueName: \"kubernetes.io/projected/33e9a2d1-1784-46f0-85e6-bd003c923d18-kube-api-access-b4hbk\") pod \"kuadrant-operator-controller-manager-86878c769f-6zgrs\" (UID: \"33e9a2d1-1784-46f0-85e6-bd003c923d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:42.926689 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.926658 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a646bbe-909c-4174-83a7-84624c2296bf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2a646bbe-909c-4174-83a7-84624c2296bf" (UID: "2a646bbe-909c-4174-83a7-84624c2296bf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:19:42.926939 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.926917 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e9a2d1-1784-46f0-85e6-bd003c923d18-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6zgrs\" (UID: \"33e9a2d1-1784-46f0-85e6-bd003c923d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:42.928364 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.928342 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a646bbe-909c-4174-83a7-84624c2296bf-kube-api-access-dfptq" (OuterVolumeSpecName: "kube-api-access-dfptq") pod "2a646bbe-909c-4174-83a7-84624c2296bf" (UID: "2a646bbe-909c-4174-83a7-84624c2296bf"). InnerVolumeSpecName "kube-api-access-dfptq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:19:42.928436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.928381 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddd9366c-a137-4080-9f1d-5a680bee2376-kube-api-access-d9lf8" (OuterVolumeSpecName: "kube-api-access-d9lf8") pod "ddd9366c-a137-4080-9f1d-5a680bee2376" (UID: "ddd9366c-a137-4080-9f1d-5a680bee2376"). InnerVolumeSpecName "kube-api-access-d9lf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:19:42.939101 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.939079 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4hbk\" (UniqueName: \"kubernetes.io/projected/33e9a2d1-1784-46f0-85e6-bd003c923d18-kube-api-access-b4hbk\") pod \"kuadrant-operator-controller-manager-86878c769f-6zgrs\" (UID: \"33e9a2d1-1784-46f0-85e6-bd003c923d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:42.960681 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.960657 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:42.967076 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:42.967061 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" Jul 09 22:19:43.017880 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.017817 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:43.027210 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.027179 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9lf8\" (UniqueName: \"kubernetes.io/projected/ddd9366c-a137-4080-9f1d-5a680bee2376-kube-api-access-d9lf8\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:19:43.027330 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.027215 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2a646bbe-909c-4174-83a7-84624c2296bf-extensions-socket-volume\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:19:43.027330 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.027234 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfptq\" (UniqueName: \"kubernetes.io/projected/2a646bbe-909c-4174-83a7-84624c2296bf-kube-api-access-dfptq\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:19:43.133047 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.133019 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd"] Jul 09 22:19:43.137394 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:19:43.137358 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacdaa1d4_cc46_40ab_8192_c8128265ed97.slice/crio-1f669727940b26bed4ebfbf5ceac9192f6810db39b4ce5265d34b6dc248ae610 WatchSource:0}: Error finding container 1f669727940b26bed4ebfbf5ceac9192f6810db39b4ce5265d34b6dc248ae610: Status 404 returned error can't find the container with id 1f669727940b26bed4ebfbf5ceac9192f6810db39b4ce5265d34b6dc248ae610 Jul 09 22:19:43.140078 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.140054 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k"] Jul 09 22:19:43.142246 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:19:43.142222 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e6fa64_5a00_43a6_9c82_b1dd5cd74ae5.slice/crio-718793adad3ee5f571aa6aa7fb5b5a42258dda83f12690567dad477cc2e9934d WatchSource:0}: Error finding container 718793adad3ee5f571aa6aa7fb5b5a42258dda83f12690567dad477cc2e9934d: Status 404 returned error can't find the container with id 718793adad3ee5f571aa6aa7fb5b5a42258dda83f12690567dad477cc2e9934d Jul 09 22:19:43.173781 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.173759 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs"] Jul 09 22:19:43.176709 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:19:43.176684 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e9a2d1_1784_46f0_85e6_bd003c923d18.slice/crio-ff320ef7d9ea4031d64edcd3b276017bcd3a17dbb8afe0aa184c40ad4db01950 WatchSource:0}: Error finding container ff320ef7d9ea4031d64edcd3b276017bcd3a17dbb8afe0aa184c40ad4db01950: Status 404 returned error can't find the container with id ff320ef7d9ea4031d64edcd3b276017bcd3a17dbb8afe0aa184c40ad4db01950 Jul 09 22:19:43.761601 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.761566 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" event={"ID":"acdaa1d4-cc46-40ab-8192-c8128265ed97","Type":"ContainerStarted","Data":"d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a"} Jul 09 22:19:43.762064 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.761629 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:43.762064 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.761651 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" event={"ID":"acdaa1d4-cc46-40ab-8192-c8128265ed97","Type":"ContainerStarted","Data":"1f669727940b26bed4ebfbf5ceac9192f6810db39b4ce5265d34b6dc248ae610"} Jul 09 22:19:43.762968 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.762952 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:43.763052 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.762958 2576 scope.go:117] "RemoveContainer" containerID="d0fa0d772dd3c161d6032519a9376f8a74e5b9a82f640269c538ea9452194b9d" Jul 09 22:19:43.764624 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.764598 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" event={"ID":"33e9a2d1-1784-46f0-85e6-bd003c923d18","Type":"ContainerStarted","Data":"77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6"} Jul 09 22:19:43.764749 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.764640 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" event={"ID":"33e9a2d1-1784-46f0-85e6-bd003c923d18","Type":"ContainerStarted","Data":"ff320ef7d9ea4031d64edcd3b276017bcd3a17dbb8afe0aa184c40ad4db01950"} Jul 09 22:19:43.764844 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:43.764823 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-999mp\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:43.764844 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.764836 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:43.765988 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.765961 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:43.767782 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.767760 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" event={"ID":"b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5","Type":"ContainerStarted","Data":"4217f3729c0abd9a0aba6cbd05b8ef84ae232ff3af5a374a94eb910cd42e2805"} Jul 09 22:19:43.767890 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.767807 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" Jul 09 22:19:43.767890 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.767830 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" event={"ID":"b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5","Type":"ContainerStarted","Data":"718793adad3ee5f571aa6aa7fb5b5a42258dda83f12690567dad477cc2e9934d"} Jul 09 22:19:43.774961 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.774937 2576 scope.go:117] "RemoveContainer" containerID="ba01bb57fdef1e44d9753ad2d86cbb7527711a83ca4c7d032c3cfda2ed1ebfa6" Jul 09 22:19:43.789120 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.789082 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" podStartSLOduration=1.789069539 podStartE2EDuration="1.789069539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:43.787517152 +0000 UTC m=+651.885732343" watchObservedRunningTime="2026-07-09 22:19:43.789069539 +0000 UTC m=+651.887284732" Jul 09 22:19:43.790288 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:43.790267 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:43.792971 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:43.792946 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-999mp\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-999mp" Jul 09 22:19:43.816119 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.816015 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" podStartSLOduration=1.815999377 podStartE2EDuration="1.815999377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:43.813835667 +0000 UTC m=+651.912050857" watchObservedRunningTime="2026-07-09 22:19:43.815999377 +0000 UTC m=+651.914214566" Jul 09 22:19:43.816894 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:43.816872 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-xzw87\" is forbidden: User \"system:node:ip-10-0-135-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-244.ec2.internal' and this object" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-xzw87" Jul 09 22:19:43.841501 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:43.841453 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" podStartSLOduration=1.841431585 podStartE2EDuration="1.841431585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:43.839578979 +0000 UTC m=+651.937794197" watchObservedRunningTime="2026-07-09 22:19:43.841431585 +0000 UTC m=+651.939646778" Jul 09 22:19:44.389647 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:44.389610 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a646bbe-909c-4174-83a7-84624c2296bf" path="/var/lib/kubelet/pods/2a646bbe-909c-4174-83a7-84624c2296bf/volumes" Jul 09 22:19:44.389958 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:44.389945 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddd9366c-a137-4080-9f1d-5a680bee2376" path="/var/lib/kubelet/pods/ddd9366c-a137-4080-9f1d-5a680bee2376/volumes" Jul 09 22:19:54.777829 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:54.777799 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:54.778318 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:54.777912 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k456k" Jul 09 22:19:54.778318 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:54.778102 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:19:54.954792 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:54.954763 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd"] Jul 09 22:19:54.954954 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:54.954929 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" podUID="acdaa1d4-cc46-40ab-8192-c8128265ed97" containerName="manager" containerID="cri-o://d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a" gracePeriod=10 Jul 09 22:19:55.202984 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.202962 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:55.330027 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.329935 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acdaa1d4-cc46-40ab-8192-c8128265ed97-extensions-socket-volume\") pod \"acdaa1d4-cc46-40ab-8192-c8128265ed97\" (UID: \"acdaa1d4-cc46-40ab-8192-c8128265ed97\") " Jul 09 22:19:55.330176 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.330057 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p76l\" (UniqueName: \"kubernetes.io/projected/acdaa1d4-cc46-40ab-8192-c8128265ed97-kube-api-access-2p76l\") pod \"acdaa1d4-cc46-40ab-8192-c8128265ed97\" (UID: \"acdaa1d4-cc46-40ab-8192-c8128265ed97\") " Jul 09 22:19:55.330350 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.330329 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdaa1d4-cc46-40ab-8192-c8128265ed97-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "acdaa1d4-cc46-40ab-8192-c8128265ed97" (UID: "acdaa1d4-cc46-40ab-8192-c8128265ed97"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:19:55.332171 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.332146 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acdaa1d4-cc46-40ab-8192-c8128265ed97-kube-api-access-2p76l" (OuterVolumeSpecName: "kube-api-access-2p76l") pod "acdaa1d4-cc46-40ab-8192-c8128265ed97" (UID: "acdaa1d4-cc46-40ab-8192-c8128265ed97"). InnerVolumeSpecName "kube-api-access-2p76l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:19:55.431398 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.431368 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acdaa1d4-cc46-40ab-8192-c8128265ed97-extensions-socket-volume\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:19:55.431398 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.431394 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p76l\" (UniqueName: \"kubernetes.io/projected/acdaa1d4-cc46-40ab-8192-c8128265ed97-kube-api-access-2p76l\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:19:55.832173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.832133 2576 generic.go:356] "Generic (PLEG): container finished" podID="acdaa1d4-cc46-40ab-8192-c8128265ed97" containerID="d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a" exitCode=0 Jul 09 22:19:55.832598 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.832180 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" event={"ID":"acdaa1d4-cc46-40ab-8192-c8128265ed97","Type":"ContainerDied","Data":"d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a"} Jul 09 22:19:55.832598 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.832189 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" Jul 09 22:19:55.832598 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.832259 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd" event={"ID":"acdaa1d4-cc46-40ab-8192-c8128265ed97","Type":"ContainerDied","Data":"1f669727940b26bed4ebfbf5ceac9192f6810db39b4ce5265d34b6dc248ae610"} Jul 09 22:19:55.832598 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.832289 2576 scope.go:117] "RemoveContainer" containerID="d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a" Jul 09 22:19:55.841492 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.841473 2576 scope.go:117] "RemoveContainer" containerID="d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a" Jul 09 22:19:55.841803 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:19:55.841782 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a\": container with ID starting with d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a not found: ID does not exist" containerID="d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a" Jul 09 22:19:55.841868 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.841813 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a"} err="failed to get container status \"d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a\": rpc error: code = NotFound desc = could not find container \"d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a\": container with ID starting with d3d299ff1a0da4d2b1d8608d2a79049b98de5758b518a0da1d429c7190bb024a not found: ID does not exist" Jul 09 22:19:55.878111 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.878060 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd"] Jul 09 22:19:55.882144 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:55.882118 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w8knd"] Jul 09 22:19:56.389400 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:19:56.389365 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acdaa1d4-cc46-40ab-8192-c8128265ed97" path="/var/lib/kubelet/pods/acdaa1d4-cc46-40ab-8192-c8128265ed97/volumes" Jul 09 22:20:11.520776 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.520740 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt"] Jul 09 22:20:11.521223 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.521160 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acdaa1d4-cc46-40ab-8192-c8128265ed97" containerName="manager" Jul 09 22:20:11.521223 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.521171 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="acdaa1d4-cc46-40ab-8192-c8128265ed97" containerName="manager" Jul 09 22:20:11.521299 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.521230 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="acdaa1d4-cc46-40ab-8192-c8128265ed97" containerName="manager" Jul 09 22:20:11.525469 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.525449 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.531166 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.531140 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-w9l85\"" Jul 09 22:20:11.531332 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.531312 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 22:20:11.539985 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.539962 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt"] Jul 09 22:20:11.561984 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.561953 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.562116 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.561991 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd5f7\" (UniqueName: \"kubernetes.io/projected/92b29da5-7df6-408f-975f-bba71829b164-kube-api-access-hd5f7\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.562116 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.562013 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.562193 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.562106 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b29da5-7df6-408f-975f-bba71829b164-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.562193 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.562136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.562193 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.562151 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b29da5-7df6-408f-975f-bba71829b164-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.562289 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.562224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b29da5-7df6-408f-975f-bba71829b164-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.562289 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.562248 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.562289 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.562278 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663269 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663233 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b29da5-7df6-408f-975f-bba71829b164-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663456 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663284 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663456 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663311 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663456 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663332 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663456 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd5f7\" (UniqueName: \"kubernetes.io/projected/92b29da5-7df6-408f-975f-bba71829b164-kube-api-access-hd5f7\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663456 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663369 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663456 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663419 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b29da5-7df6-408f-975f-bba71829b164-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663852 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663597 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663852 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663639 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b29da5-7df6-408f-975f-bba71829b164-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663959 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663851 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663959 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663889 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.663959 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.663887 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.664124 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.664102 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.664189 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.664174 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92b29da5-7df6-408f-975f-bba71829b164-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.665790 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.665772 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92b29da5-7df6-408f-975f-bba71829b164-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.665890 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.665809 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92b29da5-7df6-408f-975f-bba71829b164-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.675786 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.675758 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd5f7\" (UniqueName: \"kubernetes.io/projected/92b29da5-7df6-408f-975f-bba71829b164-kube-api-access-hd5f7\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.675893 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.675807 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92b29da5-7df6-408f-975f-bba71829b164-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-thsbt\" (UID: \"92b29da5-7df6-408f-975f-bba71829b164\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.836260 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.836232 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:11.978696 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.978665 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt"] Jul 09 22:20:11.980384 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:20:11.980356 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b29da5_7df6_408f_975f_bba71829b164.slice/crio-d968529308ba3ad2972ac5dab78d4fd381f264441032b07ae1cad4645c273a34 WatchSource:0}: Error finding container d968529308ba3ad2972ac5dab78d4fd381f264441032b07ae1cad4645c273a34: Status 404 returned error can't find the container with id d968529308ba3ad2972ac5dab78d4fd381f264441032b07ae1cad4645c273a34 Jul 09 22:20:11.982267 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:11.982251 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:20:12.912799 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:12.912764 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" event={"ID":"92b29da5-7df6-408f-975f-bba71829b164","Type":"ContainerStarted","Data":"d968529308ba3ad2972ac5dab78d4fd381f264441032b07ae1cad4645c273a34"} Jul 09 22:20:18.471006 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:18.470963 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:20:18.471384 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:18.471053 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:20:18.471384 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:18.471099 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:20:18.946250 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:18.946211 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" event={"ID":"92b29da5-7df6-408f-975f-bba71829b164","Type":"ContainerStarted","Data":"58d5071f433de1bcf52fcc7cd0daf9668463a14e6e4a36edbda6a4205b82f504"} Jul 09 22:20:18.978250 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:18.978204 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" podStartSLOduration=1.4899140339999999 podStartE2EDuration="7.978188895s" totalImagesPullingTime="6.488274861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:18.977325574 +0000 UTC m=+687.075540766" watchObservedRunningTime="2026-07-09 22:20:18.978188895 +0000 UTC m=+687.076404084" Jul 09 22:20:19.836638 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:19.836598 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:19.841332 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:19.841309 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:19.952770 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:19.952713 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:19.953649 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:19.953620 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:20:20.100793 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.100702 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j62rg"] Jul 09 22:20:20.104327 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.104304 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:20.107780 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.107754 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 22:20:20.107780 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.107774 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qxmpn\"" Jul 09 22:20:20.112932 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.112910 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j62rg"] Jul 09 22:20:20.121185 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.121164 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d770fd9d-0904-4355-ac4a-144bc2a9af1e-config-file\") pod \"limitador-limitador-64c94fd64f-j62rg\" (UID: \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:20.121296 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.121222 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7kb6\" (UniqueName: \"kubernetes.io/projected/d770fd9d-0904-4355-ac4a-144bc2a9af1e-kube-api-access-t7kb6\") pod \"limitador-limitador-64c94fd64f-j62rg\" (UID: \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:20.196093 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.196060 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j62rg"] Jul 09 22:20:20.221738 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.221684 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7kb6\" (UniqueName: \"kubernetes.io/projected/d770fd9d-0904-4355-ac4a-144bc2a9af1e-kube-api-access-t7kb6\") pod \"limitador-limitador-64c94fd64f-j62rg\" (UID: \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:20.221901 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.221745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d770fd9d-0904-4355-ac4a-144bc2a9af1e-config-file\") pod \"limitador-limitador-64c94fd64f-j62rg\" (UID: \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:20.222379 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.222360 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d770fd9d-0904-4355-ac4a-144bc2a9af1e-config-file\") pod \"limitador-limitador-64c94fd64f-j62rg\" (UID: \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:20.231128 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.231089 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7kb6\" (UniqueName: \"kubernetes.io/projected/d770fd9d-0904-4355-ac4a-144bc2a9af1e-kube-api-access-t7kb6\") pod \"limitador-limitador-64c94fd64f-j62rg\" (UID: \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:20.414897 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.414819 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:20.559480 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.559455 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j62rg"] Jul 09 22:20:20.561423 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:20:20.561399 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd770fd9d_0904_4355_ac4a_144bc2a9af1e.slice/crio-01ef2a833c833d99cd69ed510c63ef70f539549c7786fa410f1d34361b755f4e WatchSource:0}: Error finding container 01ef2a833c833d99cd69ed510c63ef70f539549c7786fa410f1d34361b755f4e: Status 404 returned error can't find the container with id 01ef2a833c833d99cd69ed510c63ef70f539549c7786fa410f1d34361b755f4e Jul 09 22:20:20.891009 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.890975 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6r9n5"] Jul 09 22:20:20.896006 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.895984 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6r9n5" Jul 09 22:20:20.899531 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.899513 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7sbg9\"" Jul 09 22:20:20.904784 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.904761 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6r9n5"] Jul 09 22:20:20.928505 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.928476 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cdrw\" (UniqueName: \"kubernetes.io/projected/b3f3b4fb-c21e-42d6-9753-a304a2ed935f-kube-api-access-2cdrw\") pod \"authorino-58cd955675-6r9n5\" (UID: \"b3f3b4fb-c21e-42d6-9753-a304a2ed935f\") " pod="kuadrant-system/authorino-58cd955675-6r9n5" Jul 09 22:20:20.958626 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:20.958585 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" event={"ID":"d770fd9d-0904-4355-ac4a-144bc2a9af1e","Type":"ContainerStarted","Data":"01ef2a833c833d99cd69ed510c63ef70f539549c7786fa410f1d34361b755f4e"} Jul 09 22:20:21.029390 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:21.029353 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cdrw\" (UniqueName: \"kubernetes.io/projected/b3f3b4fb-c21e-42d6-9753-a304a2ed935f-kube-api-access-2cdrw\") pod \"authorino-58cd955675-6r9n5\" (UID: \"b3f3b4fb-c21e-42d6-9753-a304a2ed935f\") " pod="kuadrant-system/authorino-58cd955675-6r9n5" Jul 09 22:20:21.042188 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:21.042160 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cdrw\" (UniqueName: \"kubernetes.io/projected/b3f3b4fb-c21e-42d6-9753-a304a2ed935f-kube-api-access-2cdrw\") pod \"authorino-58cd955675-6r9n5\" (UID: \"b3f3b4fb-c21e-42d6-9753-a304a2ed935f\") " pod="kuadrant-system/authorino-58cd955675-6r9n5" Jul 09 22:20:21.206348 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:21.206259 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6r9n5" Jul 09 22:20:21.360017 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:21.359987 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6r9n5"] Jul 09 22:20:21.363845 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:20:21.363479 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3f3b4fb_c21e_42d6_9753_a304a2ed935f.slice/crio-f5213f9ca3d112d571d4c3585a6d8a5cb49851d3199839b372684481a525e50a WatchSource:0}: Error finding container f5213f9ca3d112d571d4c3585a6d8a5cb49851d3199839b372684481a525e50a: Status 404 returned error can't find the container with id f5213f9ca3d112d571d4c3585a6d8a5cb49851d3199839b372684481a525e50a Jul 09 22:20:21.965535 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:21.965393 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6r9n5" event={"ID":"b3f3b4fb-c21e-42d6-9753-a304a2ed935f","Type":"ContainerStarted","Data":"f5213f9ca3d112d571d4c3585a6d8a5cb49851d3199839b372684481a525e50a"} Jul 09 22:20:23.979194 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:23.979158 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" event={"ID":"d770fd9d-0904-4355-ac4a-144bc2a9af1e","Type":"ContainerStarted","Data":"8a8bbc12c0ca3483eb02bda2cd639dad6a6f958ad9709d1a511f778d5488e3e4"} Jul 09 22:20:23.979647 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:23.979312 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:23.999813 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:23.999755 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" podStartSLOduration=1.049262917 podStartE2EDuration="3.999732876s" totalImagesPullingTime="2.950469959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:23.99794159 +0000 UTC m=+692.096156779" watchObservedRunningTime="2026-07-09 22:20:23.999732876 +0000 UTC m=+692.097948066" Jul 09 22:20:25.613747 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:25.613641 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6r9n5"] Jul 09 22:20:25.989949 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:25.989867 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6r9n5" event={"ID":"b3f3b4fb-c21e-42d6-9753-a304a2ed935f","Type":"ContainerStarted","Data":"82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e"} Jul 09 22:20:26.009307 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:26.009253 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6r9n5" podStartSLOduration=2.013197212 podStartE2EDuration="6.009237465s" totalImagesPullingTime="3.996040253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:26.007262921 +0000 UTC m=+694.105478110" watchObservedRunningTime="2026-07-09 22:20:26.009237465 +0000 UTC m=+694.107452694" Jul 09 22:20:26.994298 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:26.994258 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6r9n5" podUID="b3f3b4fb-c21e-42d6-9753-a304a2ed935f" containerName="authorino" containerID="cri-o://82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e" gracePeriod=30 Jul 09 22:20:27.251172 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:27.251121 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6r9n5" Jul 09 22:20:27.288906 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:27.288883 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cdrw\" (UniqueName: \"kubernetes.io/projected/b3f3b4fb-c21e-42d6-9753-a304a2ed935f-kube-api-access-2cdrw\") pod \"b3f3b4fb-c21e-42d6-9753-a304a2ed935f\" (UID: \"b3f3b4fb-c21e-42d6-9753-a304a2ed935f\") " Jul 09 22:20:27.290937 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:27.290909 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3f3b4fb-c21e-42d6-9753-a304a2ed935f-kube-api-access-2cdrw" (OuterVolumeSpecName: "kube-api-access-2cdrw") pod "b3f3b4fb-c21e-42d6-9753-a304a2ed935f" (UID: "b3f3b4fb-c21e-42d6-9753-a304a2ed935f"). InnerVolumeSpecName "kube-api-access-2cdrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:20:27.389645 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:27.389612 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cdrw\" (UniqueName: \"kubernetes.io/projected/b3f3b4fb-c21e-42d6-9753-a304a2ed935f-kube-api-access-2cdrw\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:20:27.999921 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:27.999886 2576 generic.go:356] "Generic (PLEG): container finished" podID="b3f3b4fb-c21e-42d6-9753-a304a2ed935f" containerID="82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e" exitCode=0 Jul 09 22:20:28.000332 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:27.999929 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6r9n5" Jul 09 22:20:28.000332 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:27.999931 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6r9n5" event={"ID":"b3f3b4fb-c21e-42d6-9753-a304a2ed935f","Type":"ContainerDied","Data":"82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e"} Jul 09 22:20:28.000332 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:27.999973 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6r9n5" event={"ID":"b3f3b4fb-c21e-42d6-9753-a304a2ed935f","Type":"ContainerDied","Data":"f5213f9ca3d112d571d4c3585a6d8a5cb49851d3199839b372684481a525e50a"} Jul 09 22:20:28.000332 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:27.999994 2576 scope.go:117] "RemoveContainer" containerID="82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e" Jul 09 22:20:28.009258 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:28.009232 2576 scope.go:117] "RemoveContainer" containerID="82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e" Jul 09 22:20:28.009539 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:20:28.009519 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e\": container with ID starting with 82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e not found: ID does not exist" containerID="82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e" Jul 09 22:20:28.009626 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:28.009547 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e"} err="failed to get container status \"82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e\": rpc error: code = NotFound desc = could not find container \"82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e\": container with ID starting with 82ab62ceb9c017544fd5096165b0624edea6b373767c06f397826a04454a4d9e not found: ID does not exist" Jul 09 22:20:28.023344 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:28.023325 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6r9n5"] Jul 09 22:20:28.027492 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:28.027475 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6r9n5"] Jul 09 22:20:28.388111 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:28.388079 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3f3b4fb-c21e-42d6-9753-a304a2ed935f" path="/var/lib/kubelet/pods/b3f3b4fb-c21e-42d6-9753-a304a2ed935f/volumes" Jul 09 22:20:34.985801 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:34.985770 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:36.603942 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:36.603906 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j62rg"] Jul 09 22:20:36.604320 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:36.604098 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" podUID="d770fd9d-0904-4355-ac4a-144bc2a9af1e" containerName="limitador" containerID="cri-o://8a8bbc12c0ca3483eb02bda2cd639dad6a6f958ad9709d1a511f778d5488e3e4" gracePeriod=30 Jul 09 22:20:37.043913 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:37.043881 2576 generic.go:356] "Generic (PLEG): container finished" podID="d770fd9d-0904-4355-ac4a-144bc2a9af1e" containerID="8a8bbc12c0ca3483eb02bda2cd639dad6a6f958ad9709d1a511f778d5488e3e4" exitCode=0 Jul 09 22:20:37.044064 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:37.043964 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" event={"ID":"d770fd9d-0904-4355-ac4a-144bc2a9af1e","Type":"ContainerDied","Data":"8a8bbc12c0ca3483eb02bda2cd639dad6a6f958ad9709d1a511f778d5488e3e4"} Jul 09 22:20:37.161175 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:37.161148 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:37.280175 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:37.280094 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d770fd9d-0904-4355-ac4a-144bc2a9af1e-config-file\") pod \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\" (UID: \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\") " Jul 09 22:20:37.280175 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:37.280156 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7kb6\" (UniqueName: \"kubernetes.io/projected/d770fd9d-0904-4355-ac4a-144bc2a9af1e-kube-api-access-t7kb6\") pod \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\" (UID: \"d770fd9d-0904-4355-ac4a-144bc2a9af1e\") " Jul 09 22:20:37.280510 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:37.280488 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d770fd9d-0904-4355-ac4a-144bc2a9af1e-config-file" (OuterVolumeSpecName: "config-file") pod "d770fd9d-0904-4355-ac4a-144bc2a9af1e" (UID: "d770fd9d-0904-4355-ac4a-144bc2a9af1e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:20:37.282155 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:37.282129 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d770fd9d-0904-4355-ac4a-144bc2a9af1e-kube-api-access-t7kb6" (OuterVolumeSpecName: "kube-api-access-t7kb6") pod "d770fd9d-0904-4355-ac4a-144bc2a9af1e" (UID: "d770fd9d-0904-4355-ac4a-144bc2a9af1e"). InnerVolumeSpecName "kube-api-access-t7kb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:20:37.381059 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:37.381026 2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d770fd9d-0904-4355-ac4a-144bc2a9af1e-config-file\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:20:37.381059 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:37.381053 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7kb6\" (UniqueName: \"kubernetes.io/projected/d770fd9d-0904-4355-ac4a-144bc2a9af1e-kube-api-access-t7kb6\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:20:38.050181 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:38.050149 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" event={"ID":"d770fd9d-0904-4355-ac4a-144bc2a9af1e","Type":"ContainerDied","Data":"01ef2a833c833d99cd69ed510c63ef70f539549c7786fa410f1d34361b755f4e"} Jul 09 22:20:38.050181 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:38.050171 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-j62rg" Jul 09 22:20:38.050646 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:38.050210 2576 scope.go:117] "RemoveContainer" containerID="8a8bbc12c0ca3483eb02bda2cd639dad6a6f958ad9709d1a511f778d5488e3e4" Jul 09 22:20:38.073316 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:38.073286 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j62rg"] Jul 09 22:20:38.077360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:38.077340 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-j62rg"] Jul 09 22:20:38.388446 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:38.388415 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d770fd9d-0904-4355-ac4a-144bc2a9af1e" path="/var/lib/kubelet/pods/d770fd9d-0904-4355-ac4a-144bc2a9af1e/volumes" Jul 09 22:20:41.787989 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.787957 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xc674"] Jul 09 22:20:41.788522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.788502 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f3b4fb-c21e-42d6-9753-a304a2ed935f" containerName="authorino" Jul 09 22:20:41.788606 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.788525 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f3b4fb-c21e-42d6-9753-a304a2ed935f" containerName="authorino" Jul 09 22:20:41.788606 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.788573 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d770fd9d-0904-4355-ac4a-144bc2a9af1e" containerName="limitador" Jul 09 22:20:41.788606 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.788583 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d770fd9d-0904-4355-ac4a-144bc2a9af1e" containerName="limitador" Jul 09 22:20:41.788787 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.788660 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d770fd9d-0904-4355-ac4a-144bc2a9af1e" containerName="limitador" Jul 09 22:20:41.788787 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.788680 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3f3b4fb-c21e-42d6-9753-a304a2ed935f" containerName="authorino" Jul 09 22:20:41.793176 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.793155 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:20:41.796660 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.796628 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 22:20:41.796813 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.796797 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 22:20:41.796870 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.796853 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 22:20:41.798688 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.798670 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-k6ln4\"" Jul 09 22:20:41.806566 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.806547 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xc674"] Jul 09 22:20:41.924582 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.924549 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw8cg\" (UniqueName: \"kubernetes.io/projected/eb5f6b04-84e0-4633-b375-d7eb6daad06e-kube-api-access-xw8cg\") pod \"postgres-69fb4594cf-xc674\" (UID: \"eb5f6b04-84e0-4633-b375-d7eb6daad06e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:20:41.924778 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:41.924592 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eb5f6b04-84e0-4633-b375-d7eb6daad06e-data\") pod \"postgres-69fb4594cf-xc674\" (UID: \"eb5f6b04-84e0-4633-b375-d7eb6daad06e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:20:42.025920 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:42.025885 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8cg\" (UniqueName: \"kubernetes.io/projected/eb5f6b04-84e0-4633-b375-d7eb6daad06e-kube-api-access-xw8cg\") pod \"postgres-69fb4594cf-xc674\" (UID: \"eb5f6b04-84e0-4633-b375-d7eb6daad06e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:20:42.025920 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:42.025924 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eb5f6b04-84e0-4633-b375-d7eb6daad06e-data\") pod \"postgres-69fb4594cf-xc674\" (UID: \"eb5f6b04-84e0-4633-b375-d7eb6daad06e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:20:42.026318 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:42.026301 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/eb5f6b04-84e0-4633-b375-d7eb6daad06e-data\") pod \"postgres-69fb4594cf-xc674\" (UID: \"eb5f6b04-84e0-4633-b375-d7eb6daad06e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:20:42.037352 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:42.037326 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw8cg\" (UniqueName: \"kubernetes.io/projected/eb5f6b04-84e0-4633-b375-d7eb6daad06e-kube-api-access-xw8cg\") pod \"postgres-69fb4594cf-xc674\" (UID: \"eb5f6b04-84e0-4633-b375-d7eb6daad06e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:20:42.103421 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:42.103392 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:20:42.281529 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:42.281501 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xc674"] Jul 09 22:20:42.282375 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:20:42.282353 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5f6b04_84e0_4633_b375_d7eb6daad06e.slice/crio-e40f4b223232754909356cc2125fdbd290b7e6c448e4490ec07e63b43b836964 WatchSource:0}: Error finding container e40f4b223232754909356cc2125fdbd290b7e6c448e4490ec07e63b43b836964: Status 404 returned error can't find the container with id e40f4b223232754909356cc2125fdbd290b7e6c448e4490ec07e63b43b836964 Jul 09 22:20:43.078021 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:43.077987 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" event={"ID":"eb5f6b04-84e0-4633-b375-d7eb6daad06e","Type":"ContainerStarted","Data":"e40f4b223232754909356cc2125fdbd290b7e6c448e4490ec07e63b43b836964"} Jul 09 22:20:48.111086 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:48.111049 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" event={"ID":"eb5f6b04-84e0-4633-b375-d7eb6daad06e","Type":"ContainerStarted","Data":"998f488c432a03dde44ec2f0112ec00e3323204fcc3df0875e086134ec730b9c"} Jul 09 22:20:48.111570 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:48.111155 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:20:48.130712 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:48.130665 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" podStartSLOduration=2.242725401 podStartE2EDuration="7.130649967s" totalImagesPullingTime="4.887924566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:48.129564972 +0000 UTC m=+716.227780162" watchObservedRunningTime="2026-07-09 22:20:48.130649967 +0000 UTC m=+716.228865156" Jul 09 22:20:54.145204 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:20:54.145177 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xc674" Jul 09 22:21:37.230497 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.230459 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-jckm7"] Jul 09 22:21:37.242797 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.242768 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:37.245089 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.245057 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-jckm7"] Jul 09 22:21:37.246874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.246850 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 22:21:37.246992 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.246850 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9dxx8\"" Jul 09 22:21:37.247745 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.247704 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-qgvd2"] Jul 09 22:21:37.251435 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.251413 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:37.261639 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.261612 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-qgvd2"] Jul 09 22:21:37.420635 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.420601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe555a7-1594-4cfa-8a16-687d9f46afad-plugins-config-volume\") pod \"payload-pre-processing-7964457696-jckm7\" (UID: \"ffe555a7-1594-4cfa-8a16-687d9f46afad\") " pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:37.420830 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.420655 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316a095c-7dd4-442f-ad2c-19edb5b7fb09-plugins-config-volume\") pod \"payload-processing-58f86c7798-qgvd2\" (UID: \"316a095c-7dd4-442f-ad2c-19edb5b7fb09\") " pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:37.420830 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.420689 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmxrv\" (UniqueName: \"kubernetes.io/projected/ffe555a7-1594-4cfa-8a16-687d9f46afad-kube-api-access-rmxrv\") pod \"payload-pre-processing-7964457696-jckm7\" (UID: \"ffe555a7-1594-4cfa-8a16-687d9f46afad\") " pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:37.420830 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.420749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5ssj\" (UniqueName: \"kubernetes.io/projected/316a095c-7dd4-442f-ad2c-19edb5b7fb09-kube-api-access-d5ssj\") pod \"payload-processing-58f86c7798-qgvd2\" (UID: \"316a095c-7dd4-442f-ad2c-19edb5b7fb09\") " pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:37.521399 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.521315 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxrv\" (UniqueName: \"kubernetes.io/projected/ffe555a7-1594-4cfa-8a16-687d9f46afad-kube-api-access-rmxrv\") pod \"payload-pre-processing-7964457696-jckm7\" (UID: \"ffe555a7-1594-4cfa-8a16-687d9f46afad\") " pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:37.521399 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.521361 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5ssj\" (UniqueName: \"kubernetes.io/projected/316a095c-7dd4-442f-ad2c-19edb5b7fb09-kube-api-access-d5ssj\") pod \"payload-processing-58f86c7798-qgvd2\" (UID: \"316a095c-7dd4-442f-ad2c-19edb5b7fb09\") " pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:37.521612 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.521405 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe555a7-1594-4cfa-8a16-687d9f46afad-plugins-config-volume\") pod \"payload-pre-processing-7964457696-jckm7\" (UID: \"ffe555a7-1594-4cfa-8a16-687d9f46afad\") " pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:37.521612 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.521436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316a095c-7dd4-442f-ad2c-19edb5b7fb09-plugins-config-volume\") pod \"payload-processing-58f86c7798-qgvd2\" (UID: \"316a095c-7dd4-442f-ad2c-19edb5b7fb09\") " pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:37.522350 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.522326 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/316a095c-7dd4-442f-ad2c-19edb5b7fb09-plugins-config-volume\") pod \"payload-processing-58f86c7798-qgvd2\" (UID: \"316a095c-7dd4-442f-ad2c-19edb5b7fb09\") " pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:37.522415 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.522326 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe555a7-1594-4cfa-8a16-687d9f46afad-plugins-config-volume\") pod \"payload-pre-processing-7964457696-jckm7\" (UID: \"ffe555a7-1594-4cfa-8a16-687d9f46afad\") " pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:37.530530 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.530503 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmxrv\" (UniqueName: \"kubernetes.io/projected/ffe555a7-1594-4cfa-8a16-687d9f46afad-kube-api-access-rmxrv\") pod \"payload-pre-processing-7964457696-jckm7\" (UID: \"ffe555a7-1594-4cfa-8a16-687d9f46afad\") " pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:37.530770 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.530625 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5ssj\" (UniqueName: \"kubernetes.io/projected/316a095c-7dd4-442f-ad2c-19edb5b7fb09-kube-api-access-d5ssj\") pod \"payload-processing-58f86c7798-qgvd2\" (UID: \"316a095c-7dd4-442f-ad2c-19edb5b7fb09\") " pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:37.554160 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.554136 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:37.564320 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.564300 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:37.717286 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.717258 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-jckm7"] Jul 09 22:21:37.719192 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:21:37.719163 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe555a7_1594_4cfa_8a16_687d9f46afad.slice/crio-a349c7820b236c57dd6454da490cf8e75ed277e8f70e55d2b4c88cfb15781e05 WatchSource:0}: Error finding container a349c7820b236c57dd6454da490cf8e75ed277e8f70e55d2b4c88cfb15781e05: Status 404 returned error can't find the container with id a349c7820b236c57dd6454da490cf8e75ed277e8f70e55d2b4c88cfb15781e05 Jul 09 22:21:37.745994 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:37.745619 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-qgvd2"] Jul 09 22:21:38.347535 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:38.347492 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" event={"ID":"316a095c-7dd4-442f-ad2c-19edb5b7fb09","Type":"ContainerStarted","Data":"5f8d2001d18b276a0a97a588f8ad64d3f520d36ed0faa3c1604585cc546c51c4"} Jul 09 22:21:38.349067 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:38.349034 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" event={"ID":"ffe555a7-1594-4cfa-8a16-687d9f46afad","Type":"ContainerStarted","Data":"a349c7820b236c57dd6454da490cf8e75ed277e8f70e55d2b4c88cfb15781e05"} Jul 09 22:21:40.364680 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:40.364645 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" event={"ID":"316a095c-7dd4-442f-ad2c-19edb5b7fb09","Type":"ContainerStarted","Data":"4d94c16397b3aef7250ae0b0f1e87d04ac65016259e06dfbb47beb5dc7831fd6"} Jul 09 22:21:40.365153 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:40.364846 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:40.366309 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:40.366285 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" event={"ID":"ffe555a7-1594-4cfa-8a16-687d9f46afad","Type":"ContainerStarted","Data":"19e876e56cd76270052ff8344e01cb99b5ff953e79852f6a78ced7554e713167"} Jul 09 22:21:40.366429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:40.366374 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:40.385110 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:40.385067 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" podStartSLOduration=1.657266362 podStartE2EDuration="3.385053187s" totalImagesPullingTime="1.727786825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:21:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:21:40.38179105 +0000 UTC m=+768.480006243" watchObservedRunningTime="2026-07-09 22:21:40.385053187 +0000 UTC m=+768.483268413" Jul 09 22:21:40.398791 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:40.398745 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" podStartSLOduration=1.641411001 podStartE2EDuration="3.39871008s" totalImagesPullingTime="1.757299079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:21:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:21:40.398261341 +0000 UTC m=+768.496476530" watchObservedRunningTime="2026-07-09 22:21:40.39871008 +0000 UTC m=+768.496925300" Jul 09 22:21:45.395446 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:45.395412 2576 generic.go:356] "Generic (PLEG): container finished" podID="92b29da5-7df6-408f-975f-bba71829b164" containerID="58d5071f433de1bcf52fcc7cd0daf9668463a14e6e4a36edbda6a4205b82f504" exitCode=0 Jul 09 22:21:45.395446 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:45.395451 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" event={"ID":"92b29da5-7df6-408f-975f-bba71829b164","Type":"ContainerDied","Data":"58d5071f433de1bcf52fcc7cd0daf9668463a14e6e4a36edbda6a4205b82f504"} Jul 09 22:21:45.395921 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:45.395837 2576 scope.go:117] "RemoveContainer" containerID="58d5071f433de1bcf52fcc7cd0daf9668463a14e6e4a36edbda6a4205b82f504" Jul 09 22:21:45.396876 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:45.396846 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:21:45.396948 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:45.396916 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:21:45.396987 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:45.396949 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:21:46.401229 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:46.401197 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" event={"ID":"92b29da5-7df6-408f-975f-bba71829b164","Type":"ContainerStarted","Data":"bfa730236a9c5e2431b1aff57c9a693825744dc061a5ef0cf5c512f0049172f7"} Jul 09 22:21:46.837351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:46.837318 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:21:46.838626 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:46.838596 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-thsbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 09 22:21:46.838874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:46.838845 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" podUID="92b29da5-7df6-408f-975f-bba71829b164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 09 22:21:47.837642 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:47.837589 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-thsbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 09 22:21:47.838187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:47.837690 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" podUID="92b29da5-7df6-408f-975f-bba71829b164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 09 22:21:48.837679 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:48.837641 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-thsbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 09 22:21:48.838313 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:48.838279 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" podUID="92b29da5-7df6-408f-975f-bba71829b164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 09 22:21:49.837525 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:49.837484 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-thsbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 09 22:21:49.837842 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:49.837812 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" podUID="92b29da5-7df6-408f-975f-bba71829b164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 09 22:21:49.953270 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:49.953238 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:21:50.840860 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:50.840830 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:21:50.841783 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:50.841763 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-thsbt" Jul 09 22:21:51.373372 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:51.373339 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-qgvd2" Jul 09 22:21:51.373566 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:51.373548 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-jckm7" Jul 09 22:21:57.472873 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.472836 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx"] Jul 09 22:21:57.477218 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.477193 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.480644 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.480621 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bf8gl\"" Jul 09 22:21:57.480779 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.480621 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 22:21:57.480779 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.480682 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 22:21:57.482518 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.482498 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 22:21:57.486426 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.486173 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx"] Jul 09 22:21:57.524855 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.524825 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.525013 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.524862 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88239bfb-290b-408a-ac13-4533bc26d694-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.525013 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.524896 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.525013 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.524973 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.525116 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.525044 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l7zf\" (UniqueName: \"kubernetes.io/projected/88239bfb-290b-408a-ac13-4533bc26d694-kube-api-access-7l7zf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.525116 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.525097 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.626107 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.626061 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.626285 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.626120 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.626285 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.626137 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88239bfb-290b-408a-ac13-4533bc26d694-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.626285 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.626165 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.626285 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.626188 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.626480 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.626340 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7zf\" (UniqueName: \"kubernetes.io/projected/88239bfb-290b-408a-ac13-4533bc26d694-kube-api-access-7l7zf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.626566 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.626542 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.626608 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.626563 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.626642 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.626619 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.628796 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.628771 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88239bfb-290b-408a-ac13-4533bc26d694-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.628937 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.628819 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88239bfb-290b-408a-ac13-4533bc26d694-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.637563 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.637536 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l7zf\" (UniqueName: \"kubernetes.io/projected/88239bfb-290b-408a-ac13-4533bc26d694-kube-api-access-7l7zf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x86dx\" (UID: \"88239bfb-290b-408a-ac13-4533bc26d694\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.790325 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.790228 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:21:57.931311 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:57.931207 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx"] Jul 09 22:21:57.936182 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:21:57.936153 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88239bfb_290b_408a_ac13_4533bc26d694.slice/crio-8a4005134ae8fc80a0abf3206d0ad69bf94c4092dfaacbf1e9ac9876ef9b5897 WatchSource:0}: Error finding container 8a4005134ae8fc80a0abf3206d0ad69bf94c4092dfaacbf1e9ac9876ef9b5897: Status 404 returned error can't find the container with id 8a4005134ae8fc80a0abf3206d0ad69bf94c4092dfaacbf1e9ac9876ef9b5897 Jul 09 22:21:58.463407 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:58.463354 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" event={"ID":"88239bfb-290b-408a-ac13-4533bc26d694","Type":"ContainerStarted","Data":"8a4005134ae8fc80a0abf3206d0ad69bf94c4092dfaacbf1e9ac9876ef9b5897"} Jul 09 22:21:59.469302 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:59.469264 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" event={"ID":"88239bfb-290b-408a-ac13-4533bc26d694","Type":"ContainerStarted","Data":"9c69580fd21baf9ee62689bade958cfda203ad18051c41ad1262a6289f79b39b"} Jul 09 22:21:59.489489 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:21:59.489440 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" podStartSLOduration=1.461414172 podStartE2EDuration="2.489423343s" totalImagesPullingTime="1.028009171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:21:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:21:59.487645106 +0000 UTC m=+787.585860296" watchObservedRunningTime="2026-07-09 22:21:59.489423343 +0000 UTC m=+787.587638533" Jul 09 22:22:06.191136 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.191105 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc"] Jul 09 22:22:06.196019 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.195994 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.199697 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.199666 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 22:22:06.206862 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.206834 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc"] Jul 09 22:22:06.304599 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.304563 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c04acfa0-7290-4b08-8fef-6d28815ea721-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.304599 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.304600 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-476jc\" (UniqueName: \"kubernetes.io/projected/c04acfa0-7290-4b08-8fef-6d28815ea721-kube-api-access-476jc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.304841 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.304630 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.304841 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.304660 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.304841 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.304760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.304841 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.304810 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.405360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.405322 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c04acfa0-7290-4b08-8fef-6d28815ea721-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.405360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.405363 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-476jc\" (UniqueName: \"kubernetes.io/projected/c04acfa0-7290-4b08-8fef-6d28815ea721-kube-api-access-476jc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.405613 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.405396 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.405613 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.405513 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.405613 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.405566 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.405613 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.405605 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.405861 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.405807 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.405915 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.405885 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.405981 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.405957 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.407743 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.407709 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c04acfa0-7290-4b08-8fef-6d28815ea721-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.407995 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.407976 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c04acfa0-7290-4b08-8fef-6d28815ea721-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.414944 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.414919 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-476jc\" (UniqueName: \"kubernetes.io/projected/c04acfa0-7290-4b08-8fef-6d28815ea721-kube-api-access-476jc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8bdxc\" (UID: \"c04acfa0-7290-4b08-8fef-6d28815ea721\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.508298 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.508209 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:06.641842 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:06.641815 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc"] Jul 09 22:22:06.643204 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:22:06.643173 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04acfa0_7290_4b08_8fef_6d28815ea721.slice/crio-920cdf2b0b3869b99dff2da69969d48ba1b3c80b0a6ed7b0399f9628f96fce69 WatchSource:0}: Error finding container 920cdf2b0b3869b99dff2da69969d48ba1b3c80b0a6ed7b0399f9628f96fce69: Status 404 returned error can't find the container with id 920cdf2b0b3869b99dff2da69969d48ba1b3c80b0a6ed7b0399f9628f96fce69 Jul 09 22:22:07.509664 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:07.509627 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" event={"ID":"c04acfa0-7290-4b08-8fef-6d28815ea721","Type":"ContainerStarted","Data":"459c202eaf0ae1d5ab119e45a3c7be93d29f49665dd4058a1cc93a1c040d1cf5"} Jul 09 22:22:07.510156 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:07.509713 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" event={"ID":"c04acfa0-7290-4b08-8fef-6d28815ea721","Type":"ContainerStarted","Data":"920cdf2b0b3869b99dff2da69969d48ba1b3c80b0a6ed7b0399f9628f96fce69"} Jul 09 22:22:07.531491 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:07.531425 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" podStartSLOduration=1.361870711 podStartE2EDuration="1.531404687s" totalImagesPullingTime="169.533976ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:22:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:22:07.529930933 +0000 UTC m=+795.628146114" watchObservedRunningTime="2026-07-09 22:22:07.531404687 +0000 UTC m=+795.629619878" Jul 09 22:22:07.791390 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:07.791308 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:22:07.791536 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:07.791437 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:22:07.803832 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:07.803811 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:22:08.525849 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:08.525818 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x86dx" Jul 09 22:22:16.509021 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:16.508939 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:16.509021 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:16.509007 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:16.521275 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:16.521250 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:16.564277 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:16.564247 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8bdxc" Jul 09 22:22:49.579612 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.579574 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl"] Jul 09 22:22:49.583511 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.583489 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.586874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.586855 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 22:22:49.594117 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.594093 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl"] Jul 09 22:22:49.709739 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.709696 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.709930 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.709776 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.709930 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.709811 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f87e661c-bdba-4493-9121-58e2dd3901d7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.709930 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.709863 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd6vp\" (UniqueName: \"kubernetes.io/projected/f87e661c-bdba-4493-9121-58e2dd3901d7-kube-api-access-pd6vp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.710074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.709938 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.710074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.710009 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.811297 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.811260 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.811297 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.811299 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.811522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.811325 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f87e661c-bdba-4493-9121-58e2dd3901d7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.811522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.811443 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd6vp\" (UniqueName: \"kubernetes.io/projected/f87e661c-bdba-4493-9121-58e2dd3901d7-kube-api-access-pd6vp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.811522 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.811509 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.811668 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.811584 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.811759 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.811666 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.811759 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.811691 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.811948 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.811924 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.813614 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.813593 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f87e661c-bdba-4493-9121-58e2dd3901d7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.813897 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.813881 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f87e661c-bdba-4493-9121-58e2dd3901d7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.820187 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.820159 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd6vp\" (UniqueName: \"kubernetes.io/projected/f87e661c-bdba-4493-9121-58e2dd3901d7-kube-api-access-pd6vp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl\" (UID: \"f87e661c-bdba-4493-9121-58e2dd3901d7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:49.894218 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:49.894180 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:50.031303 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:50.031236 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl"] Jul 09 22:22:50.032534 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:22:50.032496 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf87e661c_bdba_4493_9121_58e2dd3901d7.slice/crio-c645aba39a71a94dbdeacdc6fe36dd95ef3600714ce9a90820cc37024c570929 WatchSource:0}: Error finding container c645aba39a71a94dbdeacdc6fe36dd95ef3600714ce9a90820cc37024c570929: Status 404 returned error can't find the container with id c645aba39a71a94dbdeacdc6fe36dd95ef3600714ce9a90820cc37024c570929 Jul 09 22:22:50.715907 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:50.715870 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" event={"ID":"f87e661c-bdba-4493-9121-58e2dd3901d7","Type":"ContainerStarted","Data":"770f466cb93db54881b0fa1b51ee9c740c1d0581907a9b739da03fc01ddfd860"} Jul 09 22:22:50.716401 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:50.715921 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" event={"ID":"f87e661c-bdba-4493-9121-58e2dd3901d7","Type":"ContainerStarted","Data":"c645aba39a71a94dbdeacdc6fe36dd95ef3600714ce9a90820cc37024c570929"} Jul 09 22:22:50.737844 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:50.737790 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" podStartSLOduration=1.5558166 podStartE2EDuration="1.737775802s" totalImagesPullingTime="181.959202ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:22:50.734589929 +0000 UTC m=+838.832805121" watchObservedRunningTime="2026-07-09 22:22:50.737775802 +0000 UTC m=+838.835991033" Jul 09 22:22:59.895296 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:59.895258 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:59.895791 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:59.895334 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:22:59.907434 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:22:59.907412 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:23:00.783919 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:23:00.783884 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl" Jul 09 22:23:52.834658 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:23:52.834630 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:23:52.835916 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:23:52.835878 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:28:52.873394 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:28:52.873364 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:28:52.874986 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:28:52.874959 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:29:46.875895 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:46.875812 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs"] Jul 09 22:29:46.876394 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:46.876003 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" podUID="33e9a2d1-1784-46f0-85e6-bd003c923d18" containerName="manager" containerID="cri-o://77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6" gracePeriod=10 Jul 09 22:29:50.236472 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.236447 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:29:50.359870 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.359839 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e9a2d1-1784-46f0-85e6-bd003c923d18-extensions-socket-volume\") pod \"33e9a2d1-1784-46f0-85e6-bd003c923d18\" (UID: \"33e9a2d1-1784-46f0-85e6-bd003c923d18\") " Jul 09 22:29:50.360049 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.359938 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4hbk\" (UniqueName: \"kubernetes.io/projected/33e9a2d1-1784-46f0-85e6-bd003c923d18-kube-api-access-b4hbk\") pod \"33e9a2d1-1784-46f0-85e6-bd003c923d18\" (UID: \"33e9a2d1-1784-46f0-85e6-bd003c923d18\") " Jul 09 22:29:50.360244 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.360220 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e9a2d1-1784-46f0-85e6-bd003c923d18-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "33e9a2d1-1784-46f0-85e6-bd003c923d18" (UID: "33e9a2d1-1784-46f0-85e6-bd003c923d18"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:29:50.361878 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.361860 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e9a2d1-1784-46f0-85e6-bd003c923d18-kube-api-access-b4hbk" (OuterVolumeSpecName: "kube-api-access-b4hbk") pod "33e9a2d1-1784-46f0-85e6-bd003c923d18" (UID: "33e9a2d1-1784-46f0-85e6-bd003c923d18"). InnerVolumeSpecName "kube-api-access-b4hbk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:29:50.460657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.460629 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4hbk\" (UniqueName: \"kubernetes.io/projected/33e9a2d1-1784-46f0-85e6-bd003c923d18-kube-api-access-b4hbk\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:29:50.460657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.460655 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33e9a2d1-1784-46f0-85e6-bd003c923d18-extensions-socket-volume\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:29:50.839312 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.839280 2576 generic.go:356] "Generic (PLEG): container finished" podID="33e9a2d1-1784-46f0-85e6-bd003c923d18" containerID="77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6" exitCode=0 Jul 09 22:29:50.839466 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.839324 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" Jul 09 22:29:50.839466 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.839372 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" event={"ID":"33e9a2d1-1784-46f0-85e6-bd003c923d18","Type":"ContainerDied","Data":"77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6"} Jul 09 22:29:50.839466 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.839424 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs" event={"ID":"33e9a2d1-1784-46f0-85e6-bd003c923d18","Type":"ContainerDied","Data":"ff320ef7d9ea4031d64edcd3b276017bcd3a17dbb8afe0aa184c40ad4db01950"} Jul 09 22:29:50.839466 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.839444 2576 scope.go:117] "RemoveContainer" containerID="77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6" Jul 09 22:29:50.850598 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.850588 2576 scope.go:117] "RemoveContainer" containerID="77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6" Jul 09 22:29:50.850998 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:29:50.850981 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6\": container with ID starting with 77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6 not found: ID does not exist" containerID="77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6" Jul 09 22:29:50.851091 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.851005 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6"} err="failed to get container status \"77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6\": rpc error: code = NotFound desc = could not find container \"77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6\": container with ID starting with 77f9724d229c5717cf8905e2c2a7f74f7e77e1b34552efa51976bc4cc48f32b6 not found: ID does not exist" Jul 09 22:29:50.878040 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.878014 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs"] Jul 09 22:29:50.886833 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:50.886811 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6zgrs"] Jul 09 22:29:52.389933 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:29:52.389901 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e9a2d1-1784-46f0-85e6-bd003c923d18" path="/var/lib/kubelet/pods/33e9a2d1-1784-46f0-85e6-bd003c923d18/volumes" Jul 09 22:30:00.167986 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.167958 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk"] Jul 09 22:30:00.168350 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.168338 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e9a2d1-1784-46f0-85e6-bd003c923d18" containerName="manager" Jul 09 22:30:00.168391 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.168351 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e9a2d1-1784-46f0-85e6-bd003c923d18" containerName="manager" Jul 09 22:30:00.168436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.168427 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e9a2d1-1784-46f0-85e6-bd003c923d18" containerName="manager" Jul 09 22:30:00.172765 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.172747 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" Jul 09 22:30:00.177340 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.177320 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-57hq5\"" Jul 09 22:30:00.184546 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.184526 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk"] Jul 09 22:30:00.350644 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.350605 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5qzf\" (UniqueName: \"kubernetes.io/projected/566d8ed7-6fde-4c5f-9580-e398e0741267-kube-api-access-m5qzf\") pod \"maas-api-key-cleanup-29727270-4gfsk\" (UID: \"566d8ed7-6fde-4c5f-9580-e398e0741267\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" Jul 09 22:30:00.451928 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.451836 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5qzf\" (UniqueName: \"kubernetes.io/projected/566d8ed7-6fde-4c5f-9580-e398e0741267-kube-api-access-m5qzf\") pod \"maas-api-key-cleanup-29727270-4gfsk\" (UID: \"566d8ed7-6fde-4c5f-9580-e398e0741267\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" Jul 09 22:30:00.464609 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.464581 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5qzf\" (UniqueName: \"kubernetes.io/projected/566d8ed7-6fde-4c5f-9580-e398e0741267-kube-api-access-m5qzf\") pod \"maas-api-key-cleanup-29727270-4gfsk\" (UID: \"566d8ed7-6fde-4c5f-9580-e398e0741267\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" Jul 09 22:30:00.483401 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.483375 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" Jul 09 22:30:00.634445 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.634421 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk"] Jul 09 22:30:00.636766 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:30:00.636703 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod566d8ed7_6fde_4c5f_9580_e398e0741267.slice/crio-13ddff3bee2a14e734ff4ef743359b66e7d1aa4d6d52bd87f4d86d761650cfdc WatchSource:0}: Error finding container 13ddff3bee2a14e734ff4ef743359b66e7d1aa4d6d52bd87f4d86d761650cfdc: Status 404 returned error can't find the container with id 13ddff3bee2a14e734ff4ef743359b66e7d1aa4d6d52bd87f4d86d761650cfdc Jul 09 22:30:00.638686 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.638666 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 22:30:00.890631 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:00.890597 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" event={"ID":"566d8ed7-6fde-4c5f-9580-e398e0741267","Type":"ContainerStarted","Data":"13ddff3bee2a14e734ff4ef743359b66e7d1aa4d6d52bd87f4d86d761650cfdc"} Jul 09 22:30:03.908549 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:03.908510 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" event={"ID":"566d8ed7-6fde-4c5f-9580-e398e0741267","Type":"ContainerStarted","Data":"2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0"} Jul 09 22:30:03.931781 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:03.931733 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" podStartSLOduration=1.780269379 podStartE2EDuration="3.9317026s" totalImagesPullingTime="2.151433221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:30:03.929498929 +0000 UTC m=+1272.027714115" watchObservedRunningTime="2026-07-09 22:30:03.9317026 +0000 UTC m=+1272.029917791" Jul 09 22:30:48.071204 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.071168 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x"] Jul 09 22:30:48.076707 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.076689 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:30:48.083513 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.083490 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j5cdt\"" Jul 09 22:30:48.098887 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.098861 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x"] Jul 09 22:30:48.190474 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.190439 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1402915a-c4ae-4459-89bd-ab416e4e1dbd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tkx5x\" (UID: \"1402915a-c4ae-4459-89bd-ab416e4e1dbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:30:48.190640 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.190570 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chvpw\" (UniqueName: \"kubernetes.io/projected/1402915a-c4ae-4459-89bd-ab416e4e1dbd-kube-api-access-chvpw\") pod \"kuadrant-operator-controller-manager-86878c769f-tkx5x\" (UID: \"1402915a-c4ae-4459-89bd-ab416e4e1dbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:30:48.291145 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.291114 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1402915a-c4ae-4459-89bd-ab416e4e1dbd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tkx5x\" (UID: \"1402915a-c4ae-4459-89bd-ab416e4e1dbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:30:48.291349 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.291190 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chvpw\" (UniqueName: \"kubernetes.io/projected/1402915a-c4ae-4459-89bd-ab416e4e1dbd-kube-api-access-chvpw\") pod \"kuadrant-operator-controller-manager-86878c769f-tkx5x\" (UID: \"1402915a-c4ae-4459-89bd-ab416e4e1dbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:30:48.291535 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.291512 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1402915a-c4ae-4459-89bd-ab416e4e1dbd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tkx5x\" (UID: \"1402915a-c4ae-4459-89bd-ab416e4e1dbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:30:48.304051 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.304026 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chvpw\" (UniqueName: \"kubernetes.io/projected/1402915a-c4ae-4459-89bd-ab416e4e1dbd-kube-api-access-chvpw\") pod \"kuadrant-operator-controller-manager-86878c769f-tkx5x\" (UID: \"1402915a-c4ae-4459-89bd-ab416e4e1dbd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:30:48.386585 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.386553 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:30:48.548027 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:48.548000 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x"] Jul 09 22:30:48.548593 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:30:48.548571 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1402915a_c4ae_4459_89bd_ab416e4e1dbd.slice/crio-f4cee704bd2e0b8cfcc89863a09c89fd38151589ef740b4721dc6057fdb3cdd6 WatchSource:0}: Error finding container f4cee704bd2e0b8cfcc89863a09c89fd38151589ef740b4721dc6057fdb3cdd6: Status 404 returned error can't find the container with id f4cee704bd2e0b8cfcc89863a09c89fd38151589ef740b4721dc6057fdb3cdd6 Jul 09 22:30:49.134568 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:49.134535 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" event={"ID":"1402915a-c4ae-4459-89bd-ab416e4e1dbd","Type":"ContainerStarted","Data":"77a8a929fc147d3c97a4b82c673ce89f4516f6d7b9a4859396362f5d4962ec41"} Jul 09 22:30:49.134967 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:49.134613 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" event={"ID":"1402915a-c4ae-4459-89bd-ab416e4e1dbd","Type":"ContainerStarted","Data":"f4cee704bd2e0b8cfcc89863a09c89fd38151589ef740b4721dc6057fdb3cdd6"} Jul 09 22:30:49.134967 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:49.134635 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:30:49.187319 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:30:49.187260 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" podStartSLOduration=1.187243848 podStartE2EDuration="1.187243848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:30:49.183507853 +0000 UTC m=+1317.281723042" watchObservedRunningTime="2026-07-09 22:30:49.187243848 +0000 UTC m=+1317.285459038" Jul 09 22:31:00.011787 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:00.011753 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk"] Jul 09 22:31:00.012197 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:00.011950 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" podUID="566d8ed7-6fde-4c5f-9580-e398e0741267" containerName="cleanup" containerID="cri-o://2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0" gracePeriod=30 Jul 09 22:31:00.142127 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:00.142100 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tkx5x" Jul 09 22:31:30.206068 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.206041 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" Jul 09 22:31:30.347591 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.347553 2576 generic.go:356] "Generic (PLEG): container finished" podID="566d8ed7-6fde-4c5f-9580-e398e0741267" containerID="2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0" exitCode=137 Jul 09 22:31:30.347781 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.347605 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" Jul 09 22:31:30.347781 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.347642 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" event={"ID":"566d8ed7-6fde-4c5f-9580-e398e0741267","Type":"ContainerDied","Data":"2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0"} Jul 09 22:31:30.347781 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.347693 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk" event={"ID":"566d8ed7-6fde-4c5f-9580-e398e0741267","Type":"ContainerDied","Data":"13ddff3bee2a14e734ff4ef743359b66e7d1aa4d6d52bd87f4d86d761650cfdc"} Jul 09 22:31:30.347781 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.347739 2576 scope.go:117] "RemoveContainer" containerID="2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0" Jul 09 22:31:30.356238 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.356223 2576 scope.go:117] "RemoveContainer" containerID="2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0" Jul 09 22:31:30.356469 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:31:30.356452 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0\": container with ID starting with 2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0 not found: ID does not exist" containerID="2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0" Jul 09 22:31:30.356525 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.356478 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0"} err="failed to get container status \"2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0\": rpc error: code = NotFound desc = could not find container \"2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0\": container with ID starting with 2193891bc8e8b209617a05cd2dc5a9ced0f4bab165980319567799215b5b0ea0 not found: ID does not exist" Jul 09 22:31:30.382918 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.382897 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5qzf\" (UniqueName: \"kubernetes.io/projected/566d8ed7-6fde-4c5f-9580-e398e0741267-kube-api-access-m5qzf\") pod \"566d8ed7-6fde-4c5f-9580-e398e0741267\" (UID: \"566d8ed7-6fde-4c5f-9580-e398e0741267\") " Jul 09 22:31:30.384951 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.384922 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/566d8ed7-6fde-4c5f-9580-e398e0741267-kube-api-access-m5qzf" (OuterVolumeSpecName: "kube-api-access-m5qzf") pod "566d8ed7-6fde-4c5f-9580-e398e0741267" (UID: "566d8ed7-6fde-4c5f-9580-e398e0741267"). InnerVolumeSpecName "kube-api-access-m5qzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:31:30.483682 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.483649 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5qzf\" (UniqueName: \"kubernetes.io/projected/566d8ed7-6fde-4c5f-9580-e398e0741267-kube-api-access-m5qzf\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:31:30.666589 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.666519 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk"] Jul 09 22:31:30.670412 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:30.670388 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727270-4gfsk"] Jul 09 22:31:32.388738 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:31:32.388682 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="566d8ed7-6fde-4c5f-9580-e398e0741267" path="/var/lib/kubelet/pods/566d8ed7-6fde-4c5f-9580-e398e0741267/volumes" Jul 09 22:33:52.907944 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:33:52.907910 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:33:52.910860 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:33:52.910839 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:37:11.419702 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.419665 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr"] Jul 09 22:37:11.420144 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.420098 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="566d8ed7-6fde-4c5f-9580-e398e0741267" containerName="cleanup" Jul 09 22:37:11.420144 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.420109 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="566d8ed7-6fde-4c5f-9580-e398e0741267" containerName="cleanup" Jul 09 22:37:11.420227 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.420170 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="566d8ed7-6fde-4c5f-9580-e398e0741267" containerName="cleanup" Jul 09 22:37:11.423085 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.423061 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.426992 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.426967 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b9db9547-openshift-default-dockercfg-dc66l\"" Jul 09 22:37:11.453498 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.453465 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr"] Jul 09 22:37:11.531830 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.531779 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-data\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.532026 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.531843 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr8f9\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-kube-api-access-tr8f9\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.532026 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.531881 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-envoy\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.532026 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.531948 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-credential-socket\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.532026 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.531988 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-token\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.532026 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.532024 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-podinfo\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.532311 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.532046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-certs\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.532311 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.532093 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-socket\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.532311 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.532153 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istiod-ca-cert\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633278 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-podinfo\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633278 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633275 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-certs\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633526 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633298 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-socket\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633526 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633425 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istiod-ca-cert\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633526 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633494 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-data\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633693 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633548 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr8f9\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-kube-api-access-tr8f9\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633693 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633585 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-envoy\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633693 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633637 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-credential-socket\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633693 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633674 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-token\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633921 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633861 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-socket\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.633921 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.633861 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-data\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.634248 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.634221 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-credential-socket\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.634248 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.634239 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istiod-ca-cert\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.634419 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.634236 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-certs\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.636202 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.636165 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-podinfo\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.636419 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.636402 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-envoy\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.643582 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.643556 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-token\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.643678 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.643633 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr8f9\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-kube-api-access-tr8f9\") pod \"e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.734401 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.734305 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:11.872932 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.872902 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr"] Jul 09 22:37:11.874185 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:37:11.874159 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1614d0d6_0b1d_4a67_bdc3_aef27aca168a.slice/crio-c4efa4cc3c24f1f35b2a5a71a9afaab463cb374a0085ed262b153b44ba4abac6 WatchSource:0}: Error finding container c4efa4cc3c24f1f35b2a5a71a9afaab463cb374a0085ed262b153b44ba4abac6: Status 404 returned error can't find the container with id c4efa4cc3c24f1f35b2a5a71a9afaab463cb374a0085ed262b153b44ba4abac6 Jul 09 22:37:11.876468 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.876434 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:37:11.876563 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.876508 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:37:11.876624 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:11.876557 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:37:12.062973 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:12.062887 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" event={"ID":"1614d0d6-0b1d-4a67-bdc3-aef27aca168a","Type":"ContainerStarted","Data":"84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6"} Jul 09 22:37:12.062973 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:12.062939 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" event={"ID":"1614d0d6-0b1d-4a67-bdc3-aef27aca168a","Type":"ContainerStarted","Data":"c4efa4cc3c24f1f35b2a5a71a9afaab463cb374a0085ed262b153b44ba4abac6"} Jul 09 22:37:12.091910 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:12.091857 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" podStartSLOduration=1.091839441 podStartE2EDuration="1.091839441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:37:12.090805661 +0000 UTC m=+1700.189020851" watchObservedRunningTime="2026-07-09 22:37:12.091839441 +0000 UTC m=+1700.190054695" Jul 09 22:37:12.734977 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:12.734945 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:12.739803 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:12.739778 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:13.069216 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:13.069144 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:13.070144 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:13.070128 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:17.830329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:17.830290 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr"] Jul 09 22:37:17.830873 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:17.830547 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" podUID="1614d0d6-0b1d-4a67-bdc3-aef27aca168a" containerName="istio-proxy" containerID="cri-o://84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6" gracePeriod=30 Jul 09 22:37:23.088396 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.088370 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:23.121738 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.121631 2576 generic.go:356] "Generic (PLEG): container finished" podID="1614d0d6-0b1d-4a67-bdc3-aef27aca168a" containerID="84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6" exitCode=0 Jul 09 22:37:23.121738 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.121685 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" Jul 09 22:37:23.121947 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.121739 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" event={"ID":"1614d0d6-0b1d-4a67-bdc3-aef27aca168a","Type":"ContainerDied","Data":"84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6"} Jul 09 22:37:23.121947 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.121821 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr" event={"ID":"1614d0d6-0b1d-4a67-bdc3-aef27aca168a","Type":"ContainerDied","Data":"c4efa4cc3c24f1f35b2a5a71a9afaab463cb374a0085ed262b153b44ba4abac6"} Jul 09 22:37:23.121947 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.121847 2576 scope.go:117] "RemoveContainer" containerID="84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6" Jul 09 22:37:23.131029 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.131005 2576 scope.go:117] "RemoveContainer" containerID="84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6" Jul 09 22:37:23.131300 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:37:23.131282 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6\": container with ID starting with 84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6 not found: ID does not exist" containerID="84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6" Jul 09 22:37:23.131362 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.131313 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6"} err="failed to get container status \"84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6\": rpc error: code = NotFound desc = could not find container \"84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6\": container with ID starting with 84ac103528a70b998aad9965fa0159a990d7454dc634613cc0fe9f5ada8680c6 not found: ID does not exist" Jul 09 22:37:23.247693 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.247664 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-envoy\") pod \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " Jul 09 22:37:23.247909 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.247701 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-credential-socket\") pod \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " Jul 09 22:37:23.247909 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.247770 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-podinfo\") pod \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " Jul 09 22:37:23.247909 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.247794 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-token\") pod \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " Jul 09 22:37:23.247909 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.247812 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istiod-ca-cert\") pod \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " Jul 09 22:37:23.247909 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.247836 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-data\") pod \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " Jul 09 22:37:23.247909 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.247876 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tr8f9\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-kube-api-access-tr8f9\") pod \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " Jul 09 22:37:23.248234 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.247937 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-socket\") pod \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " Jul 09 22:37:23.248234 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.247981 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-certs\") pod \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\" (UID: \"1614d0d6-0b1d-4a67-bdc3-aef27aca168a\") " Jul 09 22:37:23.248234 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.248089 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1614d0d6-0b1d-4a67-bdc3-aef27aca168a" (UID: "1614d0d6-0b1d-4a67-bdc3-aef27aca168a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:23.248234 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.248197 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-data" (OuterVolumeSpecName: "istio-data") pod "1614d0d6-0b1d-4a67-bdc3-aef27aca168a" (UID: "1614d0d6-0b1d-4a67-bdc3-aef27aca168a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:23.248234 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.248216 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1614d0d6-0b1d-4a67-bdc3-aef27aca168a" (UID: "1614d0d6-0b1d-4a67-bdc3-aef27aca168a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:37:23.248477 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.248307 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1614d0d6-0b1d-4a67-bdc3-aef27aca168a" (UID: "1614d0d6-0b1d-4a67-bdc3-aef27aca168a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:23.248477 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.248386 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1614d0d6-0b1d-4a67-bdc3-aef27aca168a" (UID: "1614d0d6-0b1d-4a67-bdc3-aef27aca168a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:23.248477 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.248465 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-certs\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:23.248628 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.248486 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-credential-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:23.248628 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.248501 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istiod-ca-cert\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:23.248628 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.248513 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-data\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:23.250055 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.250026 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-token" (OuterVolumeSpecName: "istio-token") pod "1614d0d6-0b1d-4a67-bdc3-aef27aca168a" (UID: "1614d0d6-0b1d-4a67-bdc3-aef27aca168a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:37:23.250055 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.250036 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-kube-api-access-tr8f9" (OuterVolumeSpecName: "kube-api-access-tr8f9") pod "1614d0d6-0b1d-4a67-bdc3-aef27aca168a" (UID: "1614d0d6-0b1d-4a67-bdc3-aef27aca168a"). InnerVolumeSpecName "kube-api-access-tr8f9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:37:23.250194 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.250073 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1614d0d6-0b1d-4a67-bdc3-aef27aca168a" (UID: "1614d0d6-0b1d-4a67-bdc3-aef27aca168a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:23.250194 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.250084 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1614d0d6-0b1d-4a67-bdc3-aef27aca168a" (UID: "1614d0d6-0b1d-4a67-bdc3-aef27aca168a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 22:37:23.349279 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.349241 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-envoy\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:23.349279 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.349273 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-podinfo\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:23.349279 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.349283 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-istio-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:23.349515 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.349292 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tr8f9\" (UniqueName: \"kubernetes.io/projected/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-kube-api-access-tr8f9\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:23.349515 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.349302 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1614d0d6-0b1d-4a67-bdc3-aef27aca168a-workload-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:23.447665 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.447630 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr"] Jul 09 22:37:23.453746 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:23.453711 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b9db9547-openshift-default-58b78b5b4c-9btdr"] Jul 09 22:37:24.202801 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.202771 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8"] Jul 09 22:37:24.203218 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.203170 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1614d0d6-0b1d-4a67-bdc3-aef27aca168a" containerName="istio-proxy" Jul 09 22:37:24.203218 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.203182 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1614d0d6-0b1d-4a67-bdc3-aef27aca168a" containerName="istio-proxy" Jul 09 22:37:24.203298 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.203274 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1614d0d6-0b1d-4a67-bdc3-aef27aca168a" containerName="istio-proxy" Jul 09 22:37:24.206372 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.206355 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.210617 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.210509 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e594723c-openshift-default-dockercfg-ns8qk\"" Jul 09 22:37:24.225086 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.225058 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8"] Jul 09 22:37:24.360559 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.360518 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-podinfo\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.360770 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.360591 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-envoy\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.360770 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.360629 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-socket\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.360770 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.360651 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-credential-socket\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.360770 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.360738 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5jzc\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-kube-api-access-t5jzc\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.360996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.360771 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-certs\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.360996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.360794 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istiod-ca-cert\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.360996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.360842 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-data\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.360996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.360902 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-token\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.389347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.389320 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1614d0d6-0b1d-4a67-bdc3-aef27aca168a" path="/var/lib/kubelet/pods/1614d0d6-0b1d-4a67-bdc3-aef27aca168a/volumes" Jul 09 22:37:24.462436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-certs\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.462436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462395 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istiod-ca-cert\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.462436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462418 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-data\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.462436 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-token\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.462800 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462483 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-podinfo\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.462800 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462518 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-envoy\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.462800 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462552 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-socket\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.462800 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-credential-socket\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.462800 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462608 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5jzc\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-kube-api-access-t5jzc\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.463073 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462871 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-certs\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.463073 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.462896 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-data\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.463073 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.463041 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-socket\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.463239 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.463117 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-credential-socket\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.463415 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.463393 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istiod-ca-cert\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.464727 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.464700 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-envoy\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.464960 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.464938 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-podinfo\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.471815 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.471783 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-token\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.472529 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.472509 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5jzc\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-kube-api-access-t5jzc\") pod \"e2e-ait-e594723c-openshift-default-5969864879-bdsq8\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.517589 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.517558 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:24.664058 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.664032 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8"] Jul 09 22:37:24.666055 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:37:24.666031 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3711eab_e643_4f68_8fc3_78e1c95e5f5a.slice/crio-db9276537f4a56785e9983963017c3599cb5675cd2470e978138553f82f052da WatchSource:0}: Error finding container db9276537f4a56785e9983963017c3599cb5675cd2470e978138553f82f052da: Status 404 returned error can't find the container with id db9276537f4a56785e9983963017c3599cb5675cd2470e978138553f82f052da Jul 09 22:37:24.668319 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.668282 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:37:24.668390 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.668370 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:37:24.668464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:24.668447 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:37:25.135457 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:25.135420 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" event={"ID":"c3711eab-e643-4f68-8fc3-78e1c95e5f5a","Type":"ContainerStarted","Data":"15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25"} Jul 09 22:37:25.135624 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:25.135483 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" event={"ID":"c3711eab-e643-4f68-8fc3-78e1c95e5f5a","Type":"ContainerStarted","Data":"db9276537f4a56785e9983963017c3599cb5675cd2470e978138553f82f052da"} Jul 09 22:37:25.165989 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:25.165933 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" podStartSLOduration=1.165917081 podStartE2EDuration="1.165917081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:37:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:37:25.165630659 +0000 UTC m=+1713.263845849" watchObservedRunningTime="2026-07-09 22:37:25.165917081 +0000 UTC m=+1713.264132271" Jul 09 22:37:25.518246 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:25.518157 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:25.523470 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:25.523447 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:26.147512 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:26.147480 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:26.148545 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:26.148524 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:36.509115 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:36.509076 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8"] Jul 09 22:37:36.509569 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:36.509333 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" podUID="c3711eab-e643-4f68-8fc3-78e1c95e5f5a" containerName="istio-proxy" containerID="cri-o://15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25" gracePeriod=30 Jul 09 22:37:41.150048 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.150007 2576 patch_prober.go:28] interesting pod/e2e-ait-e594723c-openshift-default-5969864879-bdsq8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 22:37:41.150429 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.150071 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" podUID="c3711eab-e643-4f68-8fc3-78e1c95e5f5a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 22:37:41.766976 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.766912 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:41.828406 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828375 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-socket\") pod \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " Jul 09 22:37:41.828406 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828410 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-envoy\") pod \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " Jul 09 22:37:41.828647 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828451 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5jzc\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-kube-api-access-t5jzc\") pod \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " Jul 09 22:37:41.828647 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828486 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-credential-socket\") pod \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " Jul 09 22:37:41.828647 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828550 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istiod-ca-cert\") pod \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " Jul 09 22:37:41.828647 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828578 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-data\") pod \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " Jul 09 22:37:41.828647 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828618 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-certs\") pod \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " Jul 09 22:37:41.828947 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828660 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-token\") pod \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " Jul 09 22:37:41.828947 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828666 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3711eab-e643-4f68-8fc3-78e1c95e5f5a" (UID: "c3711eab-e643-4f68-8fc3-78e1c95e5f5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:41.828947 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828689 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-podinfo\") pod \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\" (UID: \"c3711eab-e643-4f68-8fc3-78e1c95e5f5a\") " Jul 09 22:37:41.828947 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.828899 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3711eab-e643-4f68-8fc3-78e1c95e5f5a" (UID: "c3711eab-e643-4f68-8fc3-78e1c95e5f5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:41.829165 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.829149 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:41.829219 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.829169 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-data\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:41.829465 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.829435 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3711eab-e643-4f68-8fc3-78e1c95e5f5a" (UID: "c3711eab-e643-4f68-8fc3-78e1c95e5f5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:41.829623 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.829596 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3711eab-e643-4f68-8fc3-78e1c95e5f5a" (UID: "c3711eab-e643-4f68-8fc3-78e1c95e5f5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:37:41.829808 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.829784 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3711eab-e643-4f68-8fc3-78e1c95e5f5a" (UID: "c3711eab-e643-4f68-8fc3-78e1c95e5f5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:41.831074 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.831042 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-kube-api-access-t5jzc" (OuterVolumeSpecName: "kube-api-access-t5jzc") pod "c3711eab-e643-4f68-8fc3-78e1c95e5f5a" (UID: "c3711eab-e643-4f68-8fc3-78e1c95e5f5a"). InnerVolumeSpecName "kube-api-access-t5jzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:37:41.831189 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.831112 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3711eab-e643-4f68-8fc3-78e1c95e5f5a" (UID: "c3711eab-e643-4f68-8fc3-78e1c95e5f5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 22:37:41.831189 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.831116 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3711eab-e643-4f68-8fc3-78e1c95e5f5a" (UID: "c3711eab-e643-4f68-8fc3-78e1c95e5f5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:41.831780 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.831755 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3711eab-e643-4f68-8fc3-78e1c95e5f5a" (UID: "c3711eab-e643-4f68-8fc3-78e1c95e5f5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:37:41.929982 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.929943 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-envoy\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:41.929982 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.929977 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5jzc\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-kube-api-access-t5jzc\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:41.929982 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.929988 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-credential-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:41.930205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.929997 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istiod-ca-cert\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:41.930205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.930008 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-workload-certs\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:41.930205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.930019 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:41.930205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:41.930027 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3711eab-e643-4f68-8fc3-78e1c95e5f5a-istio-podinfo\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:42.228956 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.228922 2576 generic.go:356] "Generic (PLEG): container finished" podID="c3711eab-e643-4f68-8fc3-78e1c95e5f5a" containerID="15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25" exitCode=0 Jul 09 22:37:42.229517 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.228967 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" Jul 09 22:37:42.229517 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.229003 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" event={"ID":"c3711eab-e643-4f68-8fc3-78e1c95e5f5a","Type":"ContainerDied","Data":"15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25"} Jul 09 22:37:42.229517 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.229056 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8" event={"ID":"c3711eab-e643-4f68-8fc3-78e1c95e5f5a","Type":"ContainerDied","Data":"db9276537f4a56785e9983963017c3599cb5675cd2470e978138553f82f052da"} Jul 09 22:37:42.229517 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.229079 2576 scope.go:117] "RemoveContainer" containerID="15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25" Jul 09 22:37:42.239136 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.239118 2576 scope.go:117] "RemoveContainer" containerID="15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25" Jul 09 22:37:42.239410 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:37:42.239390 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25\": container with ID starting with 15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25 not found: ID does not exist" containerID="15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25" Jul 09 22:37:42.239480 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.239420 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25"} err="failed to get container status \"15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25\": rpc error: code = NotFound desc = could not find container \"15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25\": container with ID starting with 15ca6210f1dfefe3ba720775320604b57bd1f044e761a7465c7e66b8f507ae25 not found: ID does not exist" Jul 09 22:37:42.254075 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.254048 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8"] Jul 09 22:37:42.260545 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.260522 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e594723c-openshift-default-5969864879-bdsq8"] Jul 09 22:37:42.388938 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.388903 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3711eab-e643-4f68-8fc3-78e1c95e5f5a" path="/var/lib/kubelet/pods/c3711eab-e643-4f68-8fc3-78e1c95e5f5a/volumes" Jul 09 22:37:42.968773 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.968741 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv"] Jul 09 22:37:42.969225 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.969207 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3711eab-e643-4f68-8fc3-78e1c95e5f5a" containerName="istio-proxy" Jul 09 22:37:42.969310 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.969228 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3711eab-e643-4f68-8fc3-78e1c95e5f5a" containerName="istio-proxy" Jul 09 22:37:42.969373 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.969322 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3711eab-e643-4f68-8fc3-78e1c95e5f5a" containerName="istio-proxy" Jul 09 22:37:42.974087 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.974064 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:42.977907 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.977883 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-b48c91c5-openshift-default-dockercfg-rhkkc\"" Jul 09 22:37:42.996517 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:42.996497 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv"] Jul 09 22:37:43.041511 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.041476 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-podinfo\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.041511 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.041510 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-data\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.041707 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.041547 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-token\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.041707 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.041636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-credential-socket\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.041707 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.041671 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-envoy\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.041707 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.041690 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-socket\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.041901 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.041766 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-certs\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.041901 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.041794 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istiod-ca-cert\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.041901 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.041813 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98v2s\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-kube-api-access-98v2s\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.142445 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142410 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-podinfo\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.142445 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142447 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-data\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.142685 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142480 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-token\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.142685 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142526 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-credential-socket\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.142685 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142556 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-envoy\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.142685 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142584 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-socket\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.142685 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142629 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-certs\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.142685 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142657 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istiod-ca-cert\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.142685 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142685 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98v2s\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-kube-api-access-98v2s\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.143095 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.142913 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-data\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.143206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.143185 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-certs\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.143330 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.143310 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-socket\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.143376 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.143327 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-credential-socket\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.143529 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.143510 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istiod-ca-cert\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.144839 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.144811 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-envoy\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.145049 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.145005 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-podinfo\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.178854 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.178813 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98v2s\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-kube-api-access-98v2s\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.179049 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.179029 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-token\") pod \"e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.285728 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.285634 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:43.434181 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.434138 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv"] Jul 09 22:37:43.434438 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:37:43.434414 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebab41e_d4cf_4ad7_a461_77ae038ea0c0.slice/crio-64f3aeea89b69bbef52acb4773a9f2f762f953e0c627ae4a1f7e03fb885fbdcd WatchSource:0}: Error finding container 64f3aeea89b69bbef52acb4773a9f2f762f953e0c627ae4a1f7e03fb885fbdcd: Status 404 returned error can't find the container with id 64f3aeea89b69bbef52acb4773a9f2f762f953e0c627ae4a1f7e03fb885fbdcd Jul 09 22:37:43.437110 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.437074 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:37:43.437203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.437165 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:37:43.437260 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:43.437206 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:37:44.252486 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:44.252451 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" event={"ID":"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0","Type":"ContainerStarted","Data":"4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69"} Jul 09 22:37:44.252664 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:44.252534 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" event={"ID":"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0","Type":"ContainerStarted","Data":"64f3aeea89b69bbef52acb4773a9f2f762f953e0c627ae4a1f7e03fb885fbdcd"} Jul 09 22:37:44.285733 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:44.285672 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" podStartSLOduration=2.28565651 podStartE2EDuration="2.28565651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:37:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:37:44.282552287 +0000 UTC m=+1732.380767489" watchObservedRunningTime="2026-07-09 22:37:44.28565651 +0000 UTC m=+1732.383871699" Jul 09 22:37:44.286691 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:44.286668 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:44.291560 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:44.291535 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:45.257403 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:45.257368 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:45.258266 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:45.258245 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:48.720210 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:48.720173 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv"] Jul 09 22:37:48.720823 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:48.720370 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" podUID="9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" containerName="istio-proxy" containerID="cri-o://4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69" gracePeriod=30 Jul 09 22:37:55.081665 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.081640 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:55.158541 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158460 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-podinfo\") pod \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " Jul 09 22:37:55.158541 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158495 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istiod-ca-cert\") pod \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " Jul 09 22:37:55.158541 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158533 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-token\") pod \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " Jul 09 22:37:55.158866 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158577 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-socket\") pod \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " Jul 09 22:37:55.158866 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158650 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-envoy\") pod \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " Jul 09 22:37:55.158866 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158673 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-certs\") pod \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " Jul 09 22:37:55.158866 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158694 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-data\") pod \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " Jul 09 22:37:55.158866 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158764 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98v2s\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-kube-api-access-98v2s\") pod \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " Jul 09 22:37:55.158866 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158806 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-credential-socket\") pod \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\" (UID: \"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0\") " Jul 09 22:37:55.159173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158919 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" (UID: "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:37:55.159173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158906 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" (UID: "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:55.159173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.158974 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" (UID: "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:55.159173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.159139 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" (UID: "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:55.159400 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.159286 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-credential-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:55.159400 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.159308 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istiod-ca-cert\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:55.159400 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.159321 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:55.159400 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.159353 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-workload-certs\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:55.159819 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.159796 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" (UID: "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:55.160732 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.160697 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" (UID: "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:37:55.161165 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.161142 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-kube-api-access-98v2s" (OuterVolumeSpecName: "kube-api-access-98v2s") pod "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" (UID: "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0"). InnerVolumeSpecName "kube-api-access-98v2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:37:55.161241 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.161176 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" (UID: "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 22:37:55.161241 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.161208 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" (UID: "9ebab41e-d4cf-4ad7-a461-77ae038ea0c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:37:55.259963 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.259930 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-envoy\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:55.259963 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.259957 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-data\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:55.259963 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.259967 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98v2s\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-kube-api-access-98v2s\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:55.260201 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.259976 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-podinfo\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:55.260201 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.259985 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0-istio-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:37:55.310596 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.310543 2576 generic.go:356] "Generic (PLEG): container finished" podID="9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" containerID="4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69" exitCode=0 Jul 09 22:37:55.310779 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.310600 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" Jul 09 22:37:55.310779 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.310614 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" event={"ID":"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0","Type":"ContainerDied","Data":"4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69"} Jul 09 22:37:55.310779 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.310676 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv" event={"ID":"9ebab41e-d4cf-4ad7-a461-77ae038ea0c0","Type":"ContainerDied","Data":"64f3aeea89b69bbef52acb4773a9f2f762f953e0c627ae4a1f7e03fb885fbdcd"} Jul 09 22:37:55.310779 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.310708 2576 scope.go:117] "RemoveContainer" containerID="4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69" Jul 09 22:37:55.320944 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.320926 2576 scope.go:117] "RemoveContainer" containerID="4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69" Jul 09 22:37:55.321212 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:37:55.321194 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69\": container with ID starting with 4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69 not found: ID does not exist" containerID="4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69" Jul 09 22:37:55.321252 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.321225 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69"} err="failed to get container status \"4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69\": rpc error: code = NotFound desc = could not find container \"4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69\": container with ID starting with 4eb408201e9f652126bfacf3a8e1bc4868d2b4d1e1fd1775de6710bc6ee63b69 not found: ID does not exist" Jul 09 22:37:55.358660 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.358626 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv"] Jul 09 22:37:55.373530 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:55.373498 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-b48c91c5-openshift-default-7b9569cdb7-trcrv"] Jul 09 22:37:56.395868 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:37:56.395836 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" path="/var/lib/kubelet/pods/9ebab41e-d4cf-4ad7-a461-77ae038ea0c0/volumes" Jul 09 22:38:52.940018 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:38:52.939991 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:38:52.944201 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:38:52.944178 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:39:32.746178 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.746143 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr"] Jul 09 22:39:32.746642 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.746582 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" containerName="istio-proxy" Jul 09 22:39:32.746642 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.746594 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" containerName="istio-proxy" Jul 09 22:39:32.746732 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.746680 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ebab41e-d4cf-4ad7-a461-77ae038ea0c0" containerName="istio-proxy" Jul 09 22:39:32.749802 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.749786 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.753599 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.753578 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b2036d22-openshift-default-dockercfg-8mjbf\"" Jul 09 22:39:32.776580 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.776546 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr"] Jul 09 22:39:32.887864 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.887819 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-token\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.887864 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.887865 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-data\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.888082 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.887887 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-certs\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.888082 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.888034 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-socket\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.888154 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.888081 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-envoy\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.888154 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.888107 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-podinfo\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.888154 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.888129 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e35dd7fd-9480-4770-90a8-83e905748ca1-istiod-ca-cert\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.888258 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.888153 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-credential-socket\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.888258 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.888214 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw88c\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-kube-api-access-zw88c\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989038 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989001 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw88c\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-kube-api-access-zw88c\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989226 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989048 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-token\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989226 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989166 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-data\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989226 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989204 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-certs\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989399 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989296 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-socket\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989399 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989342 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-envoy\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989399 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-podinfo\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989558 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989397 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e35dd7fd-9480-4770-90a8-83e905748ca1-istiod-ca-cert\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989558 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-credential-socket\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989664 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989608 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-data\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989664 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989642 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-certs\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989769 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989682 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-socket\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.989853 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.989833 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-credential-socket\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.990141 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.990116 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e35dd7fd-9480-4770-90a8-83e905748ca1-istiod-ca-cert\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.991576 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.991550 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-envoy\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.991830 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.991812 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-podinfo\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.998454 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.998406 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-token\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:32.998956 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:32.998938 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw88c\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-kube-api-access-zw88c\") pod \"e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:33.060136 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:33.060107 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:33.210698 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:33.210676 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr"] Jul 09 22:39:33.211906 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:39:33.211881 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35dd7fd_9480_4770_90a8_83e905748ca1.slice/crio-f9e3282f3130445cfd5d34bad065792fb9778d552fc723f3c5ed440ef6a08ffc WatchSource:0}: Error finding container f9e3282f3130445cfd5d34bad065792fb9778d552fc723f3c5ed440ef6a08ffc: Status 404 returned error can't find the container with id f9e3282f3130445cfd5d34bad065792fb9778d552fc723f3c5ed440ef6a08ffc Jul 09 22:39:33.214485 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:33.214452 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:39:33.214566 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:33.214519 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:39:33.214566 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:33.214547 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:39:33.807760 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:33.807711 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" event={"ID":"e35dd7fd-9480-4770-90a8-83e905748ca1","Type":"ContainerStarted","Data":"cae03e913104222314975656c0f58088e74be0a3a8701a830fc8aabfa5986393"} Jul 09 22:39:33.808255 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:33.807817 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" event={"ID":"e35dd7fd-9480-4770-90a8-83e905748ca1","Type":"ContainerStarted","Data":"f9e3282f3130445cfd5d34bad065792fb9778d552fc723f3c5ed440ef6a08ffc"} Jul 09 22:39:33.834227 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:33.834169 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" podStartSLOduration=1.834154066 podStartE2EDuration="1.834154066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:39:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:39:33.83255561 +0000 UTC m=+1841.930770801" watchObservedRunningTime="2026-07-09 22:39:33.834154066 +0000 UTC m=+1841.932369256" Jul 09 22:39:34.061126 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:34.061036 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:34.066297 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:34.066272 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:34.812815 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:34.812785 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:34.813900 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:34.813880 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:49.814381 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:49.814342 2576 patch_prober.go:28] interesting pod/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 09 22:39:49.814839 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:49.814412 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 09 22:39:49.905736 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:49.905679 2576 generic.go:356] "Generic (PLEG): container finished" podID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerID="cae03e913104222314975656c0f58088e74be0a3a8701a830fc8aabfa5986393" exitCode=0 Jul 09 22:39:49.905909 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:49.905754 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" event={"ID":"e35dd7fd-9480-4770-90a8-83e905748ca1","Type":"ContainerDied","Data":"cae03e913104222314975656c0f58088e74be0a3a8701a830fc8aabfa5986393"} Jul 09 22:39:49.906207 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:49.906190 2576 scope.go:117] "RemoveContainer" containerID="cae03e913104222314975656c0f58088e74be0a3a8701a830fc8aabfa5986393" Jul 09 22:39:49.907242 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:49.907215 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:39:49.907301 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:49.907283 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:39:49.907341 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:49.907322 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:39:50.913058 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:50.913025 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" event={"ID":"e35dd7fd-9480-4770-90a8-83e905748ca1","Type":"ContainerStarted","Data":"36046cacb174b6b443da95711d3b795370678bc0f47a47fb558f634d42baa927"} Jul 09 22:39:51.060513 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:51.060475 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:39:51.060706 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:51.060684 2576 patch_prober.go:28] interesting pod/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 09 22:39:51.060794 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:51.060769 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 09 22:39:52.061430 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:52.061393 2576 patch_prober.go:28] interesting pod/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 09 22:39:52.061886 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:52.061451 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 09 22:39:53.060969 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:53.060925 2576 patch_prober.go:28] interesting pod/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 09 22:39:53.061158 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:53.060985 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 09 22:39:54.060826 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:54.060788 2576 patch_prober.go:28] interesting pod/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 09 22:39:54.061251 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:54.060853 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 09 22:39:54.392863 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:54.392833 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr"] Jul 09 22:39:54.393065 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:54.393022 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" containerID="cri-o://36046cacb174b6b443da95711d3b795370678bc0f47a47fb558f634d42baa927" gracePeriod=30 Jul 09 22:39:59.970797 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:59.970409 2576 generic.go:356] "Generic (PLEG): container finished" podID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerID="36046cacb174b6b443da95711d3b795370678bc0f47a47fb558f634d42baa927" exitCode=0 Jul 09 22:39:59.970797 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:59.970583 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" event={"ID":"e35dd7fd-9480-4770-90a8-83e905748ca1","Type":"ContainerDied","Data":"36046cacb174b6b443da95711d3b795370678bc0f47a47fb558f634d42baa927"} Jul 09 22:39:59.970797 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:39:59.970645 2576 scope.go:117] "RemoveContainer" containerID="cae03e913104222314975656c0f58088e74be0a3a8701a830fc8aabfa5986393" Jul 09 22:40:00.051637 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.051605 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:40:00.148172 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148094 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-envoy\") pod \"e35dd7fd-9480-4770-90a8-83e905748ca1\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " Jul 09 22:40:00.148172 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148140 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw88c\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-kube-api-access-zw88c\") pod \"e35dd7fd-9480-4770-90a8-83e905748ca1\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " Jul 09 22:40:00.148411 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148193 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-certs\") pod \"e35dd7fd-9480-4770-90a8-83e905748ca1\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " Jul 09 22:40:00.148411 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148219 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e35dd7fd-9480-4770-90a8-83e905748ca1-istiod-ca-cert\") pod \"e35dd7fd-9480-4770-90a8-83e905748ca1\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " Jul 09 22:40:00.148411 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148274 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-podinfo\") pod \"e35dd7fd-9480-4770-90a8-83e905748ca1\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " Jul 09 22:40:00.148411 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148309 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-token\") pod \"e35dd7fd-9480-4770-90a8-83e905748ca1\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " Jul 09 22:40:00.148411 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148352 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-data\") pod \"e35dd7fd-9480-4770-90a8-83e905748ca1\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " Jul 09 22:40:00.148411 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148383 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-socket\") pod \"e35dd7fd-9480-4770-90a8-83e905748ca1\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " Jul 09 22:40:00.148710 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148414 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-credential-socket\") pod \"e35dd7fd-9480-4770-90a8-83e905748ca1\" (UID: \"e35dd7fd-9480-4770-90a8-83e905748ca1\") " Jul 09 22:40:00.148710 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148549 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e35dd7fd-9480-4770-90a8-83e905748ca1" (UID: "e35dd7fd-9480-4770-90a8-83e905748ca1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:40:00.148710 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148636 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e35dd7fd-9480-4770-90a8-83e905748ca1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e35dd7fd-9480-4770-90a8-83e905748ca1" (UID: "e35dd7fd-9480-4770-90a8-83e905748ca1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:40:00.148898 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148783 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e35dd7fd-9480-4770-90a8-83e905748ca1" (UID: "e35dd7fd-9480-4770-90a8-83e905748ca1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:40:00.148957 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148904 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e35dd7fd-9480-4770-90a8-83e905748ca1" (UID: "e35dd7fd-9480-4770-90a8-83e905748ca1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:40:00.149013 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148962 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-certs\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:40:00.149013 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148986 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e35dd7fd-9480-4770-90a8-83e905748ca1-istiod-ca-cert\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:40:00.149013 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.148999 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-credential-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:40:00.149410 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.149387 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-data" (OuterVolumeSpecName: "istio-data") pod "e35dd7fd-9480-4770-90a8-83e905748ca1" (UID: "e35dd7fd-9480-4770-90a8-83e905748ca1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:40:00.150446 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.150410 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-kube-api-access-zw88c" (OuterVolumeSpecName: "kube-api-access-zw88c") pod "e35dd7fd-9480-4770-90a8-83e905748ca1" (UID: "e35dd7fd-9480-4770-90a8-83e905748ca1"). InnerVolumeSpecName "kube-api-access-zw88c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:40:00.150446 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.150410 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e35dd7fd-9480-4770-90a8-83e905748ca1" (UID: "e35dd7fd-9480-4770-90a8-83e905748ca1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 22:40:00.150616 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.150602 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e35dd7fd-9480-4770-90a8-83e905748ca1" (UID: "e35dd7fd-9480-4770-90a8-83e905748ca1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:40:00.150691 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.150679 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-token" (OuterVolumeSpecName: "istio-token") pod "e35dd7fd-9480-4770-90a8-83e905748ca1" (UID: "e35dd7fd-9480-4770-90a8-83e905748ca1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:40:00.250101 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.250067 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-workload-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:40:00.250101 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.250095 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-envoy\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:40:00.250101 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.250105 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw88c\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-kube-api-access-zw88c\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:40:00.250347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.250113 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-podinfo\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:40:00.250347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.250121 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:40:00.250347 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.250129 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e35dd7fd-9480-4770-90a8-83e905748ca1-istio-data\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:40:00.976530 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.976502 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" Jul 09 22:40:00.976994 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.976554 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr" event={"ID":"e35dd7fd-9480-4770-90a8-83e905748ca1","Type":"ContainerDied","Data":"f9e3282f3130445cfd5d34bad065792fb9778d552fc723f3c5ed440ef6a08ffc"} Jul 09 22:40:00.976994 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:00.976609 2576 scope.go:117] "RemoveContainer" containerID="36046cacb174b6b443da95711d3b795370678bc0f47a47fb558f634d42baa927" Jul 09 22:40:01.003507 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:01.003479 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr"] Jul 09 22:40:01.008514 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:01.008486 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b2036d22-openshift-default-6fd6c66fdc-s6jtr"] Jul 09 22:40:02.388985 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:40:02.388952 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" path="/var/lib/kubelet/pods/e35dd7fd-9480-4770-90a8-83e905748ca1/volumes" Jul 09 22:42:54.089025 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.088992 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6"] Jul 09 22:42:54.089590 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.089570 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" Jul 09 22:42:54.089640 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.089595 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" Jul 09 22:42:54.089640 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.089615 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" Jul 09 22:42:54.089640 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.089625 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" Jul 09 22:42:54.089761 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.089731 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" Jul 09 22:42:54.089761 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.089741 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e35dd7fd-9480-4770-90a8-83e905748ca1" containerName="istio-proxy" Jul 09 22:42:54.093251 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.093232 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.097316 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.097292 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-0295ed-openshift-default-dockercfg-9t64q\"" Jul 09 22:42:54.105029 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.105008 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6"] Jul 09 22:42:54.187405 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.187370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv24h\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-kube-api-access-kv24h\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.187577 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.187414 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-envoy\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.187577 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.187481 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-token\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.187577 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.187517 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf53abc0-782a-48cd-a674-b1317cccbffc-istiod-ca-cert\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.187577 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.187548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-data\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.187763 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.187610 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-podinfo\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.187763 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.187636 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-socket\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.187763 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.187666 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-credential-socket\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.187763 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.187701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-certs\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.288642 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.288602 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-podinfo\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.288642 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.288649 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-socket\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.288920 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.288692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-credential-socket\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.288920 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.288765 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-certs\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.288920 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.288821 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv24h\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-kube-api-access-kv24h\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.288920 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.288854 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-envoy\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.288920 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.288882 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-token\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.288920 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.288908 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf53abc0-782a-48cd-a674-b1317cccbffc-istiod-ca-cert\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.289237 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.288939 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-data\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.289237 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.289117 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-socket\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.289237 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.289178 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-certs\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.289237 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.289222 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-credential-socket\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.289462 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.289351 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-data\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.289809 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.289787 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf53abc0-782a-48cd-a674-b1317cccbffc-istiod-ca-cert\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.291111 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.291085 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-envoy\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.291230 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.291214 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-podinfo\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.298297 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.298276 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-token\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.298535 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.298516 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv24h\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-kube-api-access-kv24h\") pod \"e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.404383 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.404307 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:54.549783 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.549754 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6"] Jul 09 22:42:54.551853 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:42:54.551823 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf53abc0_782a_48cd_a674_b1317cccbffc.slice/crio-8b71e01e16d7f9b9e2351909cff66ee3e2d244274d7afc5ce76f39db8758edc3 WatchSource:0}: Error finding container 8b71e01e16d7f9b9e2351909cff66ee3e2d244274d7afc5ce76f39db8758edc3: Status 404 returned error can't find the container with id 8b71e01e16d7f9b9e2351909cff66ee3e2d244274d7afc5ce76f39db8758edc3 Jul 09 22:42:54.554003 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.553951 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:42:54.554108 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.554033 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:42:54.554108 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.554068 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:42:54.851534 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.851504 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" event={"ID":"cf53abc0-782a-48cd-a674-b1317cccbffc","Type":"ContainerStarted","Data":"66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a"} Jul 09 22:42:54.851747 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.851557 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" event={"ID":"cf53abc0-782a-48cd-a674-b1317cccbffc","Type":"ContainerStarted","Data":"8b71e01e16d7f9b9e2351909cff66ee3e2d244274d7afc5ce76f39db8758edc3"} Jul 09 22:42:54.874657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:54.874608 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" podStartSLOduration=0.87458802 podStartE2EDuration="874.58802ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:42:54.873988996 +0000 UTC m=+2042.972204191" watchObservedRunningTime="2026-07-09 22:42:54.87458802 +0000 UTC m=+2042.972803211" Jul 09 22:42:55.404791 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:55.404753 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:55.409688 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:55.409664 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:55.856819 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:55.856782 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:42:55.857874 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:42:55.857846 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:43:11.898917 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:11.898878 2576 patch_prober.go:28] interesting pod/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 22:43:11.899322 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:11.898953 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 22:43:13.947643 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:13.947609 2576 generic.go:356] "Generic (PLEG): container finished" podID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerID="66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a" exitCode=0 Jul 09 22:43:13.948103 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:13.947654 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" event={"ID":"cf53abc0-782a-48cd-a674-b1317cccbffc","Type":"ContainerDied","Data":"66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a"} Jul 09 22:43:13.948103 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:13.948064 2576 scope.go:117] "RemoveContainer" containerID="66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a" Jul 09 22:43:13.949010 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:13.948983 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:43:13.949078 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:13.949052 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:43:13.949114 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:13.949083 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:43:14.958868 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:14.958812 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" event={"ID":"cf53abc0-782a-48cd-a674-b1317cccbffc","Type":"ContainerStarted","Data":"7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4"} Jul 09 22:43:15.405515 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:15.405473 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:43:15.405703 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:15.405682 2576 patch_prober.go:28] interesting pod/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 09 22:43:15.405798 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:15.405761 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 09 22:43:16.404806 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:16.404768 2576 patch_prober.go:28] interesting pod/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 09 22:43:16.405202 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:16.404823 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 09 22:43:17.405303 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:17.405261 2576 patch_prober.go:28] interesting pod/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 09 22:43:17.405696 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:17.405327 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 09 22:43:18.405490 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:18.405459 2576 patch_prober.go:28] interesting pod/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 09 22:43:18.405935 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:18.405512 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 09 22:43:19.408282 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:19.408250 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:43:19.408744 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:19.408556 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:43:19.409171 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:19.409151 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:43:52.974607 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:52.974578 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:43:52.984751 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:43:52.984708 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:45:08.558515 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:08.558439 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6"] Jul 09 22:45:08.559036 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:08.558635 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" containerID="cri-o://7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4" gracePeriod=30 Jul 09 22:45:13.814789 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.814761 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:45:13.848533 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848507 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-certs\") pod \"cf53abc0-782a-48cd-a674-b1317cccbffc\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " Jul 09 22:45:13.848533 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848541 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf53abc0-782a-48cd-a674-b1317cccbffc-istiod-ca-cert\") pod \"cf53abc0-782a-48cd-a674-b1317cccbffc\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " Jul 09 22:45:13.848881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848563 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-credential-socket\") pod \"cf53abc0-782a-48cd-a674-b1317cccbffc\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " Jul 09 22:45:13.848881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848598 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-data\") pod \"cf53abc0-782a-48cd-a674-b1317cccbffc\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " Jul 09 22:45:13.848881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848751 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-socket\") pod \"cf53abc0-782a-48cd-a674-b1317cccbffc\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " Jul 09 22:45:13.848881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848778 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-token\") pod \"cf53abc0-782a-48cd-a674-b1317cccbffc\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " Jul 09 22:45:13.848881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848810 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-podinfo\") pod \"cf53abc0-782a-48cd-a674-b1317cccbffc\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " Jul 09 22:45:13.848881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848832 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf53abc0-782a-48cd-a674-b1317cccbffc" (UID: "cf53abc0-782a-48cd-a674-b1317cccbffc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:13.848881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848840 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-envoy\") pod \"cf53abc0-782a-48cd-a674-b1317cccbffc\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " Jul 09 22:45:13.848881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.848867 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kv24h\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-kube-api-access-kv24h\") pod \"cf53abc0-782a-48cd-a674-b1317cccbffc\" (UID: \"cf53abc0-782a-48cd-a674-b1317cccbffc\") " Jul 09 22:45:13.849286 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.849005 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf53abc0-782a-48cd-a674-b1317cccbffc" (UID: "cf53abc0-782a-48cd-a674-b1317cccbffc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:13.849286 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.849021 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf53abc0-782a-48cd-a674-b1317cccbffc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf53abc0-782a-48cd-a674-b1317cccbffc" (UID: "cf53abc0-782a-48cd-a674-b1317cccbffc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:45:13.849286 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.849223 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-credential-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:13.849286 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.849241 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-certs\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:13.849468 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.849334 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf53abc0-782a-48cd-a674-b1317cccbffc" (UID: "cf53abc0-782a-48cd-a674-b1317cccbffc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:13.849468 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.849255 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf53abc0-782a-48cd-a674-b1317cccbffc-istiod-ca-cert\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:13.849600 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.849575 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf53abc0-782a-48cd-a674-b1317cccbffc" (UID: "cf53abc0-782a-48cd-a674-b1317cccbffc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:13.851150 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.851121 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf53abc0-782a-48cd-a674-b1317cccbffc" (UID: "cf53abc0-782a-48cd-a674-b1317cccbffc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:13.851454 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.851427 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf53abc0-782a-48cd-a674-b1317cccbffc" (UID: "cf53abc0-782a-48cd-a674-b1317cccbffc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 22:45:13.851454 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.851439 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-kube-api-access-kv24h" (OuterVolumeSpecName: "kube-api-access-kv24h") pod "cf53abc0-782a-48cd-a674-b1317cccbffc" (UID: "cf53abc0-782a-48cd-a674-b1317cccbffc"). InnerVolumeSpecName "kube-api-access-kv24h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:45:13.851614 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.851498 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf53abc0-782a-48cd-a674-b1317cccbffc" (UID: "cf53abc0-782a-48cd-a674-b1317cccbffc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:45:13.950184 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.950147 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-data\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:13.950184 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.950176 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-workload-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:13.950184 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.950185 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:13.950184 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.950195 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-podinfo\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:13.950451 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.950203 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf53abc0-782a-48cd-a674-b1317cccbffc-istio-envoy\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:13.950451 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:13.950212 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kv24h\" (UniqueName: \"kubernetes.io/projected/cf53abc0-782a-48cd-a674-b1317cccbffc-kube-api-access-kv24h\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:14.581146 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.581111 2576 generic.go:356] "Generic (PLEG): container finished" podID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerID="7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4" exitCode=0 Jul 09 22:45:14.581336 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.581160 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" Jul 09 22:45:14.581336 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.581197 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" event={"ID":"cf53abc0-782a-48cd-a674-b1317cccbffc","Type":"ContainerDied","Data":"7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4"} Jul 09 22:45:14.581336 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.581255 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6" event={"ID":"cf53abc0-782a-48cd-a674-b1317cccbffc","Type":"ContainerDied","Data":"8b71e01e16d7f9b9e2351909cff66ee3e2d244274d7afc5ce76f39db8758edc3"} Jul 09 22:45:14.581336 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.581285 2576 scope.go:117] "RemoveContainer" containerID="7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4" Jul 09 22:45:14.590304 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.590281 2576 scope.go:117] "RemoveContainer" containerID="66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a" Jul 09 22:45:14.598326 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.598309 2576 scope.go:117] "RemoveContainer" containerID="7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4" Jul 09 22:45:14.598604 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:45:14.598585 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4\": container with ID starting with 7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4 not found: ID does not exist" containerID="7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4" Jul 09 22:45:14.598664 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.598612 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4"} err="failed to get container status \"7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4\": rpc error: code = NotFound desc = could not find container \"7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4\": container with ID starting with 7083d7815f01fa0df2b9eb0abe6880b82248bf12d1d49f84efa8c7d800ec70f4 not found: ID does not exist" Jul 09 22:45:14.598664 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.598629 2576 scope.go:117] "RemoveContainer" containerID="66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a" Jul 09 22:45:14.598911 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:45:14.598894 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a\": container with ID starting with 66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a not found: ID does not exist" containerID="66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a" Jul 09 22:45:14.598979 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.598920 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a"} err="failed to get container status \"66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a\": rpc error: code = NotFound desc = could not find container \"66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a\": container with ID starting with 66dbca9d2ee1628ff75f334c18542cd44fbf007b7f7addef48cd969355bc081a not found: ID does not exist" Jul 09 22:45:14.602270 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.602251 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6"] Jul 09 22:45:14.608205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:14.608187 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-0295ed-openshift-default-6cf6c7cd75-h7sq6"] Jul 09 22:45:16.391996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.391877 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" path="/var/lib/kubelet/pods/cf53abc0-782a-48cd-a674-b1317cccbffc/volumes" Jul 09 22:45:16.724107 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.724025 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg"] Jul 09 22:45:16.724449 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.724436 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" Jul 09 22:45:16.724504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.724452 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" Jul 09 22:45:16.724504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.724462 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" Jul 09 22:45:16.724504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.724468 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" Jul 09 22:45:16.724610 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.724557 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" Jul 09 22:45:16.724610 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.724564 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf53abc0-782a-48cd-a674-b1317cccbffc" containerName="istio-proxy" Jul 09 22:45:16.729262 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.729237 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.733747 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.733695 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ce834b-openshift-default-dockercfg-ltvfb\"" Jul 09 22:45:16.748395 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.748372 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg"] Jul 09 22:45:16.774188 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.774155 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-envoy\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.774188 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.774188 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-token\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.774408 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.774213 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-socket\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.774408 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.774279 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-credential-socket\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.774408 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.774328 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-certs\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.774408 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.774384 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-data\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.774408 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.774405 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-podinfo\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.774624 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.774426 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1be3152-5527-411f-b584-4c8fba64a9c8-istiod-ca-cert\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.774624 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.774515 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtvdl\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-kube-api-access-mtvdl\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.875985 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.875951 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtvdl\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-kube-api-access-mtvdl\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876178 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.875994 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-envoy\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876178 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876015 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-token\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876178 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876138 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-socket\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876188 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-credential-socket\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876206 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-certs\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876266 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-data\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876287 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-podinfo\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876316 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1be3152-5527-411f-b584-4c8fba64a9c8-istiod-ca-cert\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876615 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876542 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-socket\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876615 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876572 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-credential-socket\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876748 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876646 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-data\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876748 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876668 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-certs\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.876967 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.876949 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1be3152-5527-411f-b584-4c8fba64a9c8-istiod-ca-cert\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.878357 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.878334 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-envoy\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.878621 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.878600 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-podinfo\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.884867 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.884844 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-token\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:16.885549 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:16.885528 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtvdl\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-kube-api-access-mtvdl\") pod \"e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:17.039328 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:17.039238 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:17.197732 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:17.197691 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg"] Jul 09 22:45:17.198521 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:45:17.198493 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1be3152_5527_411f_b584_4c8fba64a9c8.slice/crio-2f9f90ab6f87ad344f7c18a0eeae6d2391d7d6798203e993b92467c49a78f74c WatchSource:0}: Error finding container 2f9f90ab6f87ad344f7c18a0eeae6d2391d7d6798203e993b92467c49a78f74c: Status 404 returned error can't find the container with id 2f9f90ab6f87ad344f7c18a0eeae6d2391d7d6798203e993b92467c49a78f74c Jul 09 22:45:17.200535 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:17.200505 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:45:17.200592 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:17.200568 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:45:17.200629 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:17.200596 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:45:17.600202 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:17.600170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" event={"ID":"f1be3152-5527-411f-b584-4c8fba64a9c8","Type":"ContainerStarted","Data":"14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef"} Jul 09 22:45:17.600675 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:17.600233 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" event={"ID":"f1be3152-5527-411f-b584-4c8fba64a9c8","Type":"ContainerStarted","Data":"2f9f90ab6f87ad344f7c18a0eeae6d2391d7d6798203e993b92467c49a78f74c"} Jul 09 22:45:17.625301 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:17.625253 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" podStartSLOduration=1.6252385 podStartE2EDuration="1.6252385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:45:17.624901734 +0000 UTC m=+2185.723116935" watchObservedRunningTime="2026-07-09 22:45:17.6252385 +0000 UTC m=+2185.723453689" Jul 09 22:45:18.039450 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:18.039371 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:18.044408 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:18.044383 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:18.605795 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:18.605754 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:18.606843 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:18.606825 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:34.647910 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:34.647856 2576 patch_prober.go:28] interesting pod/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 22:45:34.648326 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:34.647936 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 22:45:35.695825 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:35.695790 2576 generic.go:356] "Generic (PLEG): container finished" podID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerID="14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef" exitCode=0 Jul 09 22:45:35.696364 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:35.695875 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" event={"ID":"f1be3152-5527-411f-b584-4c8fba64a9c8","Type":"ContainerDied","Data":"14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef"} Jul 09 22:45:35.696433 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:35.696413 2576 scope.go:117] "RemoveContainer" containerID="14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef" Jul 09 22:45:35.697421 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:35.697392 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:45:35.697528 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:35.697476 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:45:35.697596 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:35.697525 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:45:36.702553 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:36.702518 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" event={"ID":"f1be3152-5527-411f-b584-4c8fba64a9c8","Type":"ContainerStarted","Data":"ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8"} Jul 09 22:45:37.041148 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:37.040319 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:37.041148 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:37.040782 2576 patch_prober.go:28] interesting pod/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 09 22:45:37.041148 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:37.040833 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 09 22:45:38.040226 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:38.040180 2576 patch_prober.go:28] interesting pod/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 09 22:45:38.040770 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:38.040248 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 09 22:45:39.040084 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:39.040044 2576 patch_prober.go:28] interesting pod/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 09 22:45:39.040410 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:39.040378 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 09 22:45:40.040238 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:40.040199 2576 patch_prober.go:28] interesting pod/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 09 22:45:40.040644 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:40.040612 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 09 22:45:41.055152 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:41.055117 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:41.055566 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:41.055514 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:41.056125 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:41.056108 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:46.974712 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:46.974671 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg"] Jul 09 22:45:46.975218 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:46.974927 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" containerID="cri-o://ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8" gracePeriod=30 Jul 09 22:45:52.231894 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.231869 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:52.321914 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.321820 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-token\") pod \"f1be3152-5527-411f-b584-4c8fba64a9c8\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " Jul 09 22:45:52.321914 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.321872 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-socket\") pod \"f1be3152-5527-411f-b584-4c8fba64a9c8\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " Jul 09 22:45:52.321914 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.321909 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-podinfo\") pod \"f1be3152-5527-411f-b584-4c8fba64a9c8\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " Jul 09 22:45:52.322173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.321926 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtvdl\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-kube-api-access-mtvdl\") pod \"f1be3152-5527-411f-b584-4c8fba64a9c8\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " Jul 09 22:45:52.322173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.321982 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-data\") pod \"f1be3152-5527-411f-b584-4c8fba64a9c8\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " Jul 09 22:45:52.322173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.322045 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1be3152-5527-411f-b584-4c8fba64a9c8-istiod-ca-cert\") pod \"f1be3152-5527-411f-b584-4c8fba64a9c8\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " Jul 09 22:45:52.322173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.322077 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-credential-socket\") pod \"f1be3152-5527-411f-b584-4c8fba64a9c8\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " Jul 09 22:45:52.322173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.322110 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-certs\") pod \"f1be3152-5527-411f-b584-4c8fba64a9c8\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " Jul 09 22:45:52.322173 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.322152 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-envoy\") pod \"f1be3152-5527-411f-b584-4c8fba64a9c8\" (UID: \"f1be3152-5527-411f-b584-4c8fba64a9c8\") " Jul 09 22:45:52.322499 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.322244 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1be3152-5527-411f-b584-4c8fba64a9c8" (UID: "f1be3152-5527-411f-b584-4c8fba64a9c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:52.322499 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.322427 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1be3152-5527-411f-b584-4c8fba64a9c8" (UID: "f1be3152-5527-411f-b584-4c8fba64a9c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:52.322616 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.322499 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:52.322616 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.322489 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1be3152-5527-411f-b584-4c8fba64a9c8" (UID: "f1be3152-5527-411f-b584-4c8fba64a9c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:52.323013 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.322983 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1be3152-5527-411f-b584-4c8fba64a9c8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1be3152-5527-411f-b584-4c8fba64a9c8" (UID: "f1be3152-5527-411f-b584-4c8fba64a9c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:45:52.323241 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.323224 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1be3152-5527-411f-b584-4c8fba64a9c8" (UID: "f1be3152-5527-411f-b584-4c8fba64a9c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:52.324153 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.324128 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1be3152-5527-411f-b584-4c8fba64a9c8" (UID: "f1be3152-5527-411f-b584-4c8fba64a9c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 22:45:52.324265 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.324200 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1be3152-5527-411f-b584-4c8fba64a9c8" (UID: "f1be3152-5527-411f-b584-4c8fba64a9c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:45:52.324403 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.324385 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1be3152-5527-411f-b584-4c8fba64a9c8" (UID: "f1be3152-5527-411f-b584-4c8fba64a9c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:45:52.324468 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.324400 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-kube-api-access-mtvdl" (OuterVolumeSpecName: "kube-api-access-mtvdl") pod "f1be3152-5527-411f-b584-4c8fba64a9c8" (UID: "f1be3152-5527-411f-b584-4c8fba64a9c8"). InnerVolumeSpecName "kube-api-access-mtvdl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:45:52.423139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.423108 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1be3152-5527-411f-b584-4c8fba64a9c8-istiod-ca-cert\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:52.423139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.423135 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-credential-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:52.423139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.423144 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-workload-certs\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:52.423351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.423153 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-envoy\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:52.423351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.423160 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:52.423351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.423169 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-podinfo\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:52.423351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.423177 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtvdl\" (UniqueName: \"kubernetes.io/projected/f1be3152-5527-411f-b584-4c8fba64a9c8-kube-api-access-mtvdl\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:52.423351 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.423185 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1be3152-5527-411f-b584-4c8fba64a9c8-istio-data\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:45:52.792458 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.792420 2576 generic.go:356] "Generic (PLEG): container finished" podID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerID="ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8" exitCode=0 Jul 09 22:45:52.792654 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.792480 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" Jul 09 22:45:52.792654 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.792502 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" event={"ID":"f1be3152-5527-411f-b584-4c8fba64a9c8","Type":"ContainerDied","Data":"ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8"} Jul 09 22:45:52.792654 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.792588 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg" event={"ID":"f1be3152-5527-411f-b584-4c8fba64a9c8","Type":"ContainerDied","Data":"2f9f90ab6f87ad344f7c18a0eeae6d2391d7d6798203e993b92467c49a78f74c"} Jul 09 22:45:52.792654 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.792616 2576 scope.go:117] "RemoveContainer" containerID="ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8" Jul 09 22:45:52.801363 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.801345 2576 scope.go:117] "RemoveContainer" containerID="14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef" Jul 09 22:45:52.808916 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.808901 2576 scope.go:117] "RemoveContainer" containerID="ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8" Jul 09 22:45:52.809170 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:45:52.809154 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8\": container with ID starting with ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8 not found: ID does not exist" containerID="ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8" Jul 09 22:45:52.809227 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.809177 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8"} err="failed to get container status \"ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8\": rpc error: code = NotFound desc = could not find container \"ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8\": container with ID starting with ef1fef1912fd37d876d0f81d5c589511110c5d23dd51a736bb68f77332d2f6d8 not found: ID does not exist" Jul 09 22:45:52.809227 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.809192 2576 scope.go:117] "RemoveContainer" containerID="14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef" Jul 09 22:45:52.809427 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:45:52.809406 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef\": container with ID starting with 14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef not found: ID does not exist" containerID="14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef" Jul 09 22:45:52.809484 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.809437 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef"} err="failed to get container status \"14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef\": rpc error: code = NotFound desc = could not find container \"14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef\": container with ID starting with 14fcf3e9889a8eb7210fa7d905bbc0c2cf60a2e5bfde9484e831dbbbca9c6eef not found: ID does not exist" Jul 09 22:45:52.820193 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.820170 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg"] Jul 09 22:45:52.825737 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:52.825705 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ce834b-openshift-default-5dc9bbd4b9-kk2fg"] Jul 09 22:45:54.388435 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:54.388403 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" path="/var/lib/kubelet/pods/f1be3152-5527-411f-b584-4c8fba64a9c8/volumes" Jul 09 22:45:55.134026 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.133981 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88"] Jul 09 22:45:55.134405 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.134393 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" Jul 09 22:45:55.134454 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.134407 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" Jul 09 22:45:55.134504 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.134490 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" Jul 09 22:45:55.139897 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.139874 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.143940 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.143919 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a70a31-openshift-default-dockercfg-kdhk8\"" Jul 09 22:45:55.147134 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.147107 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-istio-token\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.147223 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.147167 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d74bx\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-kube-api-access-d74bx\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.147283 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.147227 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a749723-340b-467c-85f2-72d8a090bc1c-istiod-ca-cert\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.147341 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.147284 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-data\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.147400 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.147337 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-envoy\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.147400 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.147389 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a749723-340b-467c-85f2-72d8a090bc1c-istio-podinfo\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.147481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.147432 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-certs\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.147517 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.147474 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-credential-socket\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.147551 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.147512 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-socket\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.154592 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.154566 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88"] Jul 09 22:45:55.248598 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.248562 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d74bx\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-kube-api-access-d74bx\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.248784 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.248623 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a749723-340b-467c-85f2-72d8a090bc1c-istiod-ca-cert\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.248784 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.248659 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-data\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.248784 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.248699 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-envoy\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.248996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.248787 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a749723-340b-467c-85f2-72d8a090bc1c-istio-podinfo\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.248996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.248820 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-certs\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.248996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.248847 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-credential-socket\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.248996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.248874 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-socket\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.248996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.248923 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-istio-token\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.249260 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.249133 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-data\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.249350 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.249320 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-credential-socket\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.249350 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.249346 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-socket\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.249549 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.249344 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-certs\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.249549 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.249503 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a749723-340b-467c-85f2-72d8a090bc1c-istiod-ca-cert\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.251014 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.250993 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-envoy\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.251236 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.251220 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a749723-340b-467c-85f2-72d8a090bc1c-istio-podinfo\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.257914 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.257897 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-istio-token\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.258205 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.258186 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d74bx\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-kube-api-access-d74bx\") pod \"e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.451072 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.450974 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:55.618620 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:45:55.618583 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a749723_340b_467c_85f2_72d8a090bc1c.slice/crio-98aeb523b2a6ce3b0e311973ceff61c22c5b5c16ed8093cadbeb6356ffae1680 WatchSource:0}: Error finding container 98aeb523b2a6ce3b0e311973ceff61c22c5b5c16ed8093cadbeb6356ffae1680: Status 404 returned error can't find the container with id 98aeb523b2a6ce3b0e311973ceff61c22c5b5c16ed8093cadbeb6356ffae1680 Jul 09 22:45:55.621345 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.621311 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:45:55.621451 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.621394 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:45:55.621451 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.621439 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:45:55.623830 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.623804 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88"] Jul 09 22:45:55.810618 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.810513 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" event={"ID":"0a749723-340b-467c-85f2-72d8a090bc1c","Type":"ContainerStarted","Data":"8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e"} Jul 09 22:45:55.810618 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.810579 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" event={"ID":"0a749723-340b-467c-85f2-72d8a090bc1c","Type":"ContainerStarted","Data":"98aeb523b2a6ce3b0e311973ceff61c22c5b5c16ed8093cadbeb6356ffae1680"} Jul 09 22:45:55.858082 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:55.858034 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" podStartSLOduration=0.858018729 podStartE2EDuration="858.018729ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:45:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:45:55.857380322 +0000 UTC m=+2223.955595514" watchObservedRunningTime="2026-07-09 22:45:55.858018729 +0000 UTC m=+2223.956233952" Jul 09 22:45:56.452044 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:56.452009 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:56.456383 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:56.456360 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:56.816268 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:56.816190 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:45:56.817030 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:45:56.817014 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:46:02.900360 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:02.900325 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx"] Jul 09 22:46:02.900961 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:02.900939 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" Jul 09 22:46:02.900961 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:02.900958 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" Jul 09 22:46:02.901130 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:02.901082 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1be3152-5527-411f-b584-4c8fba64a9c8" containerName="istio-proxy" Jul 09 22:46:02.904700 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:02.904680 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:02.908530 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:02.908510 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-d3fb60-openshift-default-dockercfg-b49k6\"" Jul 09 22:46:02.918408 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:02.918384 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx"] Jul 09 22:46:03.026501 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.026468 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkqtp\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-kube-api-access-dkqtp\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.026501 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.026505 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-data\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.026705 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.026533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9455cc-da86-4e18-939b-4cb80842b6b9-istiod-ca-cert\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.026705 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.026600 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-envoy\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.026705 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.026640 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-credential-socket\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.026705 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.026664 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-certs\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.026705 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.026701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-socket\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.026928 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.026822 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-podinfo\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.026928 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.026861 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-token\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.128305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.128266 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9455cc-da86-4e18-939b-4cb80842b6b9-istiod-ca-cert\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.128493 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.128316 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-envoy\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.128493 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.128423 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-credential-socket\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.128493 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.128456 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-certs\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.128657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.128499 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-socket\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.128657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.128589 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-podinfo\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.128657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.128639 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-token\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.128836 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.128675 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkqtp\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-kube-api-access-dkqtp\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.128836 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.128705 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-data\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.129057 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.129033 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-data\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.129158 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.129134 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-credential-socket\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.129330 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.129136 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9455cc-da86-4e18-939b-4cb80842b6b9-istiod-ca-cert\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.129464 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.129436 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-socket\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.129560 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.129534 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-certs\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.131250 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.131228 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-podinfo\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.131663 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.131632 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-envoy\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.138668 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.138644 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-token\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.139310 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.139289 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkqtp\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-kube-api-access-dkqtp\") pod \"e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.216801 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.216703 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:03.361167 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.361141 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx"] Jul 09 22:46:03.362387 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:46:03.362359 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9455cc_da86_4e18_939b_4cb80842b6b9.slice/crio-e16dd8d1bcad1bb1e0a7a9c5ef6ac6a6a34a19fbe9d838a101002dfdaef2f227 WatchSource:0}: Error finding container e16dd8d1bcad1bb1e0a7a9c5ef6ac6a6a34a19fbe9d838a101002dfdaef2f227: Status 404 returned error can't find the container with id e16dd8d1bcad1bb1e0a7a9c5ef6ac6a6a34a19fbe9d838a101002dfdaef2f227 Jul 09 22:46:03.364624 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.364592 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:46:03.364687 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.364657 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:46:03.364742 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.364684 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:46:03.863083 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.863046 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" event={"ID":"8b9455cc-da86-4e18-939b-4cb80842b6b9","Type":"ContainerStarted","Data":"b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd"} Jul 09 22:46:03.863295 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.863109 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" event={"ID":"8b9455cc-da86-4e18-939b-4cb80842b6b9","Type":"ContainerStarted","Data":"e16dd8d1bcad1bb1e0a7a9c5ef6ac6a6a34a19fbe9d838a101002dfdaef2f227"} Jul 09 22:46:03.888923 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:03.888875 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" podStartSLOduration=1.888860759 podStartE2EDuration="1.888860759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:46:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:46:03.887061506 +0000 UTC m=+2231.985276700" watchObservedRunningTime="2026-07-09 22:46:03.888860759 +0000 UTC m=+2231.987076011" Jul 09 22:46:04.217553 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:04.217469 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:04.222266 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:04.222241 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:04.869022 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:04.868994 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:04.869920 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:04.869888 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:11.817851 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:11.817812 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 09 22:46:11.818275 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:11.817886 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 09 22:46:11.904168 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:11.904138 2576 generic.go:356] "Generic (PLEG): container finished" podID="0a749723-340b-467c-85f2-72d8a090bc1c" containerID="8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e" exitCode=0 Jul 09 22:46:11.904338 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:11.904206 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" event={"ID":"0a749723-340b-467c-85f2-72d8a090bc1c","Type":"ContainerDied","Data":"8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e"} Jul 09 22:46:11.904617 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:11.904604 2576 scope.go:117] "RemoveContainer" containerID="8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e" Jul 09 22:46:11.905578 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:11.905550 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:46:11.905649 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:11.905622 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:46:11.905686 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:11.905658 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:46:12.910877 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:12.910845 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" event={"ID":"0a749723-340b-467c-85f2-72d8a090bc1c","Type":"ContainerStarted","Data":"2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf"} Jul 09 22:46:13.451617 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:13.451576 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:46:13.451851 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:13.451738 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 09 22:46:13.451851 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:13.451782 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 09 22:46:14.452227 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:14.452115 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 09 22:46:14.452227 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:14.452189 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 09 22:46:15.452176 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:15.452141 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 09 22:46:15.452455 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:15.452428 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 09 22:46:16.452422 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:16.451940 2576 patch_prober.go:28] interesting pod/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 09 22:46:16.452422 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:16.452009 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 09 22:46:17.455219 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:17.455182 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:46:17.455639 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:17.455617 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:46:17.456319 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:17.456301 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:46:18.948017 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:18.947983 2576 generic.go:356] "Generic (PLEG): container finished" podID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerID="b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd" exitCode=0 Jul 09 22:46:18.948442 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:18.948055 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" event={"ID":"8b9455cc-da86-4e18-939b-4cb80842b6b9","Type":"ContainerDied","Data":"b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd"} Jul 09 22:46:18.948609 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:18.948593 2576 scope.go:117] "RemoveContainer" containerID="b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd" Jul 09 22:46:18.949510 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:18.949486 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:46:18.949563 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:18.949549 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:46:18.949600 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:18.949584 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 09 22:46:19.869520 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:19.869487 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:19.966378 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:19.966338 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" event={"ID":"8b9455cc-da86-4e18-939b-4cb80842b6b9","Type":"ContainerStarted","Data":"342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179"} Jul 09 22:46:20.217435 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:20.217352 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:20.218513 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:20.218489 2576 patch_prober.go:28] interesting pod/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 09 22:46:20.218611 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:20.218539 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 09 22:46:21.218091 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:21.218050 2576 patch_prober.go:28] interesting pod/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 09 22:46:21.218481 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:21.218111 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 09 22:46:22.217266 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:22.217236 2576 patch_prober.go:28] interesting pod/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 09 22:46:22.217442 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:22.217304 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 09 22:46:23.217565 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:23.217529 2576 patch_prober.go:28] interesting pod/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 09 22:46:23.217986 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:23.217580 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 09 22:46:24.221420 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:24.221384 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:24.221851 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:24.221754 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:46:24.222328 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:46:24.222307 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:48:53.010487 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:48:53.010450 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:48:53.018030 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:48:53.018007 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:52:54.530004 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:54.529961 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88"] Jul 09 22:52:54.530624 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:54.530211 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" containerID="cri-o://2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf" gracePeriod=30 Jul 09 22:52:59.777679 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.777655 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:52:59.879104 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879072 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-envoy\") pod \"0a749723-340b-467c-85f2-72d8a090bc1c\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " Jul 09 22:52:59.879313 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879139 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-istio-token\") pod \"0a749723-340b-467c-85f2-72d8a090bc1c\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " Jul 09 22:52:59.879313 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879170 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-data\") pod \"0a749723-340b-467c-85f2-72d8a090bc1c\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " Jul 09 22:52:59.879313 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879207 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-credential-socket\") pod \"0a749723-340b-467c-85f2-72d8a090bc1c\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " Jul 09 22:52:59.879490 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879349 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-certs\") pod \"0a749723-340b-467c-85f2-72d8a090bc1c\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " Jul 09 22:52:59.879490 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879420 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a749723-340b-467c-85f2-72d8a090bc1c-istio-podinfo\") pod \"0a749723-340b-467c-85f2-72d8a090bc1c\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " Jul 09 22:52:59.879490 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879459 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d74bx\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-kube-api-access-d74bx\") pod \"0a749723-340b-467c-85f2-72d8a090bc1c\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " Jul 09 22:52:59.879657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879498 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a749723-340b-467c-85f2-72d8a090bc1c" (UID: "0a749723-340b-467c-85f2-72d8a090bc1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:52:59.879657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879505 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-socket\") pod \"0a749723-340b-467c-85f2-72d8a090bc1c\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " Jul 09 22:52:59.879657 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879616 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a749723-340b-467c-85f2-72d8a090bc1c" (UID: "0a749723-340b-467c-85f2-72d8a090bc1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:52:59.879850 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879709 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a749723-340b-467c-85f2-72d8a090bc1c" (UID: "0a749723-340b-467c-85f2-72d8a090bc1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:52:59.879977 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.879953 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a749723-340b-467c-85f2-72d8a090bc1c-istiod-ca-cert\") pod \"0a749723-340b-467c-85f2-72d8a090bc1c\" (UID: \"0a749723-340b-467c-85f2-72d8a090bc1c\") " Jul 09 22:52:59.880352 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.880325 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a749723-340b-467c-85f2-72d8a090bc1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a749723-340b-467c-85f2-72d8a090bc1c" (UID: "0a749723-340b-467c-85f2-72d8a090bc1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:52:59.880439 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.880334 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-credential-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:52:59.880439 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.880375 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a749723-340b-467c-85f2-72d8a090bc1c" (UID: "0a749723-340b-467c-85f2-72d8a090bc1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:52:59.880439 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.880385 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-certs\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:52:59.880439 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.880411 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-workload-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:52:59.881605 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.881579 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a749723-340b-467c-85f2-72d8a090bc1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a749723-340b-467c-85f2-72d8a090bc1c" (UID: "0a749723-340b-467c-85f2-72d8a090bc1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 22:52:59.881681 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.881622 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-kube-api-access-d74bx" (OuterVolumeSpecName: "kube-api-access-d74bx") pod "0a749723-340b-467c-85f2-72d8a090bc1c" (UID: "0a749723-340b-467c-85f2-72d8a090bc1c"). InnerVolumeSpecName "kube-api-access-d74bx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:52:59.881775 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.881743 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a749723-340b-467c-85f2-72d8a090bc1c" (UID: "0a749723-340b-467c-85f2-72d8a090bc1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:52:59.881775 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.881760 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a749723-340b-467c-85f2-72d8a090bc1c" (UID: "0a749723-340b-467c-85f2-72d8a090bc1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:52:59.981551 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.981513 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a749723-340b-467c-85f2-72d8a090bc1c-istiod-ca-cert\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:52:59.981551 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.981548 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-envoy\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:52:59.981551 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.981557 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-istio-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:52:59.981819 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.981564 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a749723-340b-467c-85f2-72d8a090bc1c-istio-data\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:52:59.981819 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.981573 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a749723-340b-467c-85f2-72d8a090bc1c-istio-podinfo\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:52:59.981819 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:52:59.981582 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d74bx\" (UniqueName: \"kubernetes.io/projected/0a749723-340b-467c-85f2-72d8a090bc1c-kube-api-access-d74bx\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:00.054816 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.054782 2576 generic.go:356] "Generic (PLEG): container finished" podID="0a749723-340b-467c-85f2-72d8a090bc1c" containerID="2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf" exitCode=0 Jul 09 22:53:00.054994 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.054836 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" Jul 09 22:53:00.054994 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.054875 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" event={"ID":"0a749723-340b-467c-85f2-72d8a090bc1c","Type":"ContainerDied","Data":"2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf"} Jul 09 22:53:00.054994 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.054931 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88" event={"ID":"0a749723-340b-467c-85f2-72d8a090bc1c","Type":"ContainerDied","Data":"98aeb523b2a6ce3b0e311973ceff61c22c5b5c16ed8093cadbeb6356ffae1680"} Jul 09 22:53:00.054994 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.054953 2576 scope.go:117] "RemoveContainer" containerID="2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf" Jul 09 22:53:00.064985 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.064968 2576 scope.go:117] "RemoveContainer" containerID="8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e" Jul 09 22:53:00.073335 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.073318 2576 scope.go:117] "RemoveContainer" containerID="2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf" Jul 09 22:53:00.073596 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:53:00.073578 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf\": container with ID starting with 2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf not found: ID does not exist" containerID="2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf" Jul 09 22:53:00.073642 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.073604 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf"} err="failed to get container status \"2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf\": rpc error: code = NotFound desc = could not find container \"2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf\": container with ID starting with 2add890cc7cb7b659e7606a37864a4d8605e4d1cda387a9b42520d283131fdbf not found: ID does not exist" Jul 09 22:53:00.073642 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.073620 2576 scope.go:117] "RemoveContainer" containerID="8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e" Jul 09 22:53:00.073887 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:53:00.073869 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e\": container with ID starting with 8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e not found: ID does not exist" containerID="8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e" Jul 09 22:53:00.073936 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.073893 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e"} err="failed to get container status \"8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e\": rpc error: code = NotFound desc = could not find container \"8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e\": container with ID starting with 8aff7640ec13917cc07eee667d62c53a50edaa0f1a2a3166573f6a320bdb604e not found: ID does not exist" Jul 09 22:53:00.082196 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.082174 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88"] Jul 09 22:53:00.088353 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.088334 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a70a31-openshift-default-5f676b549f-8bc88"] Jul 09 22:53:00.389026 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:00.388995 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" path="/var/lib/kubelet/pods/0a749723-340b-467c-85f2-72d8a090bc1c/volumes" Jul 09 22:53:01.636284 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:01.636254 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx"] Jul 09 22:53:01.636704 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:01.636478 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" containerID="cri-o://342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179" gracePeriod=30 Jul 09 22:53:06.890981 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.890957 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:53:06.953599 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953512 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkqtp\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-kube-api-access-dkqtp\") pod \"8b9455cc-da86-4e18-939b-4cb80842b6b9\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " Jul 09 22:53:06.953599 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953554 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-envoy\") pod \"8b9455cc-da86-4e18-939b-4cb80842b6b9\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " Jul 09 22:53:06.953599 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953575 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-credential-socket\") pod \"8b9455cc-da86-4e18-939b-4cb80842b6b9\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " Jul 09 22:53:06.953599 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953598 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9455cc-da86-4e18-939b-4cb80842b6b9-istiod-ca-cert\") pod \"8b9455cc-da86-4e18-939b-4cb80842b6b9\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " Jul 09 22:53:06.954019 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953618 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-data\") pod \"8b9455cc-da86-4e18-939b-4cb80842b6b9\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " Jul 09 22:53:06.954019 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953681 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-token\") pod \"8b9455cc-da86-4e18-939b-4cb80842b6b9\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " Jul 09 22:53:06.954019 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953708 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-socket\") pod \"8b9455cc-da86-4e18-939b-4cb80842b6b9\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " Jul 09 22:53:06.954019 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953779 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-podinfo\") pod \"8b9455cc-da86-4e18-939b-4cb80842b6b9\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " Jul 09 22:53:06.954019 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953820 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-certs\") pod \"8b9455cc-da86-4e18-939b-4cb80842b6b9\" (UID: \"8b9455cc-da86-4e18-939b-4cb80842b6b9\") " Jul 09 22:53:06.954019 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.953913 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b9455cc-da86-4e18-939b-4cb80842b6b9" (UID: "8b9455cc-da86-4e18-939b-4cb80842b6b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:53:06.954329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.954067 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b9455cc-da86-4e18-939b-4cb80842b6b9" (UID: "8b9455cc-da86-4e18-939b-4cb80842b6b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:53:06.954329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.954104 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b9455cc-da86-4e18-939b-4cb80842b6b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b9455cc-da86-4e18-939b-4cb80842b6b9" (UID: "8b9455cc-da86-4e18-939b-4cb80842b6b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 22:53:06.954329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.954157 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b9455cc-da86-4e18-939b-4cb80842b6b9" (UID: "8b9455cc-da86-4e18-939b-4cb80842b6b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:53:06.954329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.954225 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-credential-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:06.954329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.954242 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b9455cc-da86-4e18-939b-4cb80842b6b9-istiod-ca-cert\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:06.954329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.954254 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-socket\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:06.954329 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.954267 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-workload-certs\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:06.954694 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.954495 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b9455cc-da86-4e18-939b-4cb80842b6b9" (UID: "8b9455cc-da86-4e18-939b-4cb80842b6b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:53:06.955867 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.955837 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-kube-api-access-dkqtp" (OuterVolumeSpecName: "kube-api-access-dkqtp") pod "8b9455cc-da86-4e18-939b-4cb80842b6b9" (UID: "8b9455cc-da86-4e18-939b-4cb80842b6b9"). InnerVolumeSpecName "kube-api-access-dkqtp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:53:06.955998 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.955899 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b9455cc-da86-4e18-939b-4cb80842b6b9" (UID: "8b9455cc-da86-4e18-939b-4cb80842b6b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 22:53:06.956125 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.956104 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b9455cc-da86-4e18-939b-4cb80842b6b9" (UID: "8b9455cc-da86-4e18-939b-4cb80842b6b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 22:53:06.956344 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:06.956328 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b9455cc-da86-4e18-939b-4cb80842b6b9" (UID: "8b9455cc-da86-4e18-939b-4cb80842b6b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 22:53:07.054907 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.054869 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-token\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:07.054907 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.054899 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-podinfo\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:07.054907 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.054909 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkqtp\" (UniqueName: \"kubernetes.io/projected/8b9455cc-da86-4e18-939b-4cb80842b6b9-kube-api-access-dkqtp\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:07.055133 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.054918 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-envoy\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:07.055133 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.054927 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b9455cc-da86-4e18-939b-4cb80842b6b9-istio-data\") on node \"ip-10-0-135-244.ec2.internal\" DevicePath \"\"" Jul 09 22:53:07.094027 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.093996 2576 generic.go:356] "Generic (PLEG): container finished" podID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerID="342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179" exitCode=0 Jul 09 22:53:07.094184 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.094047 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" Jul 09 22:53:07.094184 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.094071 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" event={"ID":"8b9455cc-da86-4e18-939b-4cb80842b6b9","Type":"ContainerDied","Data":"342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179"} Jul 09 22:53:07.094184 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.094142 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx" event={"ID":"8b9455cc-da86-4e18-939b-4cb80842b6b9","Type":"ContainerDied","Data":"e16dd8d1bcad1bb1e0a7a9c5ef6ac6a6a34a19fbe9d838a101002dfdaef2f227"} Jul 09 22:53:07.094184 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.094164 2576 scope.go:117] "RemoveContainer" containerID="342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179" Jul 09 22:53:07.106140 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.106097 2576 scope.go:117] "RemoveContainer" containerID="b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd" Jul 09 22:53:07.114577 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.114558 2576 scope.go:117] "RemoveContainer" containerID="342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179" Jul 09 22:53:07.114836 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:53:07.114818 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179\": container with ID starting with 342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179 not found: ID does not exist" containerID="342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179" Jul 09 22:53:07.114884 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.114846 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179"} err="failed to get container status \"342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179\": rpc error: code = NotFound desc = could not find container \"342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179\": container with ID starting with 342ddedc1dd0bcdd2c9fdca7f0c628894f7d7899c3e5e2c7f15aa22a04e96179 not found: ID does not exist" Jul 09 22:53:07.114884 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.114862 2576 scope.go:117] "RemoveContainer" containerID="b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd" Jul 09 22:53:07.115071 ip-10-0-135-244 kubenswrapper[2576]: E0709 22:53:07.115055 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd\": container with ID starting with b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd not found: ID does not exist" containerID="b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd" Jul 09 22:53:07.115106 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.115078 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd"} err="failed to get container status \"b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd\": rpc error: code = NotFound desc = could not find container \"b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd\": container with ID starting with b6ad48578bb1301aba3366f46eca570f410ff2341e48528f7af8d82d84d959fd not found: ID does not exist" Jul 09 22:53:07.124632 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.124610 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx"] Jul 09 22:53:07.129305 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.129284 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-d3fb60-openshift-default-db6d6ddc5-9frkx"] Jul 09 22:53:07.150948 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.150931 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-chzl4_fb70b5d3-8bde-4df1-90f4-fee384cdad1e/manager/0.log" Jul 09 22:53:07.271881 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.271815 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xpd77_59df5409-675c-4c7f-9f90-cd474d423f75/manager/0.log" Jul 09 22:53:07.513759 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.513736 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-22mwk_9aa9a7d2-8530-4b4b-b0f4-35828b196c89/server/0.log" Jul 09 22:53:07.629273 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:07.629237 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b5p8s_4481c981-a256-41e3-a1dc-23eddfe664a4/manager/0.log" Jul 09 22:53:08.007960 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:08.007882 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b79867d65-kt8xm_a0fdc908-ab18-4b60-9d33-8e272264b19e/copy-manifests/0.log" Jul 09 22:53:08.015122 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:08.015103 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b79867d65-kt8xm_a0fdc908-ab18-4b60-9d33-8e272264b19e/manager/0.log" Jul 09 22:53:08.388849 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:08.388818 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" path="/var/lib/kubelet/pods/8b9455cc-da86-4e18-939b-4cb80842b6b9/volumes" Jul 09 22:53:08.474622 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:08.474593 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-xc674_eb5f6b04-84e0-4633-b375-d7eb6daad06e/postgres/0.log" Jul 09 22:53:09.987784 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:09.987755 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5jtdz_1fcbc8f6-1604-4d51-81c0-1b6111e23358/manager/0.log" Jul 09 22:53:10.105136 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:10.105106 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-h8kcp_e4a4d0b2-e041-45c1-8505-e3d08827d853/manager/0.log" Jul 09 22:53:10.337180 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:10.337157 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mfbnm_b5a6d0b1-5192-41b6-94c4-abcd18e90091/registry-server/0.log" Jul 09 22:53:10.454965 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:10.454939 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tkx5x_1402915a-c4ae-4459-89bd-ab416e4e1dbd/manager/0.log" Jul 09 22:53:10.705241 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:10.705163 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k456k_b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5/manager/0.log" Jul 09 22:53:11.412880 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:11.412843 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ff5d987cc-rf44x_c7605dd3-5b80-465f-912f-19b91742c54b/kube-auth-proxy/0.log" Jul 09 22:53:11.533757 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:11.533710 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thsbt_92b29da5-7df6-408f-975f-bba71829b164/istio-proxy/1.log" Jul 09 22:53:11.654458 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:11.654430 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jckm7_ffe555a7-1594-4cfa-8a16-687d9f46afad/payload-pre-processing/0.log" Jul 09 22:53:11.771708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:11.771634 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qgvd2_316a095c-7dd4-442f-ad2c-19edb5b7fb09/payload-processing/0.log" Jul 09 22:53:11.894031 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:11.894004 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58c6fd45c7-x2b2x_e28edf89-6b16-4538-97f4-7f72e368a48d/router/0.log" Jul 09 22:53:12.363407 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:12.363383 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-x86dx_88239bfb-290b-408a-ac13-4533bc26d694/main/0.log" Jul 09 22:53:12.472763 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:12.472736 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8bdxc_c04acfa0-7290-4b08-8fef-6d28815ea721/main/0.log" Jul 09 22:53:12.821617 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:12.821535 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-l75jl_f87e661c-bdba-4493-9121-58e2dd3901d7/main/0.log" Jul 09 22:53:19.591880 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:19.591848 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vm5vl_b276e595-42b4-452f-8c2b-703a2f894f07/global-pull-secret-syncer/0.log" Jul 09 22:53:19.737072 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:19.737043 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w75wq_4d976efd-a2f2-4a0e-a75c-f58ad1a4b5d5/konnectivity-agent/0.log" Jul 09 22:53:19.793203 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:19.793176 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-244.ec2.internal_9528662dba682bac361080b2cce7b1c7/haproxy/0.log" Jul 09 22:53:25.415215 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:25.415182 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5jtdz_1fcbc8f6-1604-4d51-81c0-1b6111e23358/manager/0.log" Jul 09 22:53:25.444609 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:25.444584 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-h8kcp_e4a4d0b2-e041-45c1-8505-e3d08827d853/manager/0.log" Jul 09 22:53:25.522004 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:25.521943 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-mfbnm_b5a6d0b1-5192-41b6-94c4-abcd18e90091/registry-server/0.log" Jul 09 22:53:25.627654 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:25.627625 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tkx5x_1402915a-c4ae-4459-89bd-ab416e4e1dbd/manager/0.log" Jul 09 22:53:25.779521 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:25.779435 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k456k_b3e6fa64-5a00-43a6-9c82-b1dd5cd74ae5/manager/0.log" Jul 09 22:53:27.908789 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:27.908759 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5gtgk_f446a306-a346-4583-9fa7-d5e505eb47d7/kube-state-metrics/0.log" Jul 09 22:53:27.934546 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:27.934517 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5gtgk_f446a306-a346-4583-9fa7-d5e505eb47d7/kube-rbac-proxy-main/0.log" Jul 09 22:53:27.961371 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:27.961347 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5gtgk_f446a306-a346-4583-9fa7-d5e505eb47d7/kube-rbac-proxy-self/0.log" Jul 09 22:53:28.254528 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.254500 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sb7k5_187f0a72-40a2-47c9-b5ab-a203ca59ff58/node-exporter/0.log" Jul 09 22:53:28.283991 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.283967 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sb7k5_187f0a72-40a2-47c9-b5ab-a203ca59ff58/kube-rbac-proxy/0.log" Jul 09 22:53:28.312319 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.312292 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sb7k5_187f0a72-40a2-47c9-b5ab-a203ca59ff58/init-textfile/0.log" Jul 09 22:53:28.439698 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.439668 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a910b6d9-c228-481d-8242-d630fab89363/prometheus/0.log" Jul 09 22:53:28.469850 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.469821 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a910b6d9-c228-481d-8242-d630fab89363/config-reloader/0.log" Jul 09 22:53:28.498987 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.498967 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a910b6d9-c228-481d-8242-d630fab89363/thanos-sidecar/0.log" Jul 09 22:53:28.525620 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.525559 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a910b6d9-c228-481d-8242-d630fab89363/kube-rbac-proxy-web/0.log" Jul 09 22:53:28.550987 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.550964 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a910b6d9-c228-481d-8242-d630fab89363/kube-rbac-proxy/0.log" Jul 09 22:53:28.581044 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.581016 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a910b6d9-c228-481d-8242-d630fab89363/kube-rbac-proxy-thanos/0.log" Jul 09 22:53:28.609078 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.609057 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a910b6d9-c228-481d-8242-d630fab89363/init-config-reloader/0.log" Jul 09 22:53:28.830529 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.830447 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd449898-8rg6h_52ce14be-4c7b-49f3-b018-8841249939c9/thanos-query/0.log" Jul 09 22:53:28.860897 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.860874 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd449898-8rg6h_52ce14be-4c7b-49f3-b018-8841249939c9/kube-rbac-proxy-web/0.log" Jul 09 22:53:28.884188 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.884163 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd449898-8rg6h_52ce14be-4c7b-49f3-b018-8841249939c9/kube-rbac-proxy/0.log" Jul 09 22:53:28.919578 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.919549 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd449898-8rg6h_52ce14be-4c7b-49f3-b018-8841249939c9/prom-label-proxy/0.log" Jul 09 22:53:28.949711 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.949688 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd449898-8rg6h_52ce14be-4c7b-49f3-b018-8841249939c9/kube-rbac-proxy-rules/0.log" Jul 09 22:53:28.976873 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:28.976847 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bcd449898-8rg6h_52ce14be-4c7b-49f3-b018-8841249939c9/kube-rbac-proxy-metrics/0.log" Jul 09 22:53:30.259380 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.259347 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh"] Jul 09 22:53:30.260021 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.259999 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" Jul 09 22:53:30.260071 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260025 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" Jul 09 22:53:30.260071 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260038 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" Jul 09 22:53:30.260071 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260046 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" Jul 09 22:53:30.260165 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260083 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" Jul 09 22:53:30.260165 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260093 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" Jul 09 22:53:30.260165 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260103 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" Jul 09 22:53:30.260165 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260112 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" Jul 09 22:53:30.260292 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260196 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a749723-340b-467c-85f2-72d8a090bc1c" containerName="istio-proxy" Jul 09 22:53:30.260292 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260204 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" Jul 09 22:53:30.260292 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.260211 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9455cc-da86-4e18-939b-4cb80842b6b9" containerName="istio-proxy" Jul 09 22:53:30.263470 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.263454 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.271463 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.271443 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t249s\"/\"kube-root-ca.crt\"" Jul 09 22:53:30.273175 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.273151 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh"] Jul 09 22:53:30.273525 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.273506 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t249s\"/\"default-dockercfg-5q4rk\"" Jul 09 22:53:30.273629 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.273603 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t249s\"/\"openshift-service-ca.crt\"" Jul 09 22:53:30.363154 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.363121 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-lib-modules\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.363154 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.363166 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-sys\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.363412 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.363189 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-podres\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.363412 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.363292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-proc\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.363412 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.363327 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5gph\" (UniqueName: \"kubernetes.io/projected/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-kube-api-access-x5gph\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.463973 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.463926 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-lib-modules\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.463973 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.463977 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-sys\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.464197 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.463997 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-podres\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.464197 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.464059 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-proc\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.464197 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.464068 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-sys\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.464197 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.464084 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5gph\" (UniqueName: \"kubernetes.io/projected/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-kube-api-access-x5gph\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.464197 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.464113 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-podres\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.464197 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.464117 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-lib-modules\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.464197 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.464171 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-proc\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.474840 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.474815 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5gph\" (UniqueName: \"kubernetes.io/projected/75c8fc9a-fe24-432e-bec5-c3c50e60b81a-kube-api-access-x5gph\") pod \"perf-node-gather-daemonset-sh4fh\" (UID: \"75c8fc9a-fe24-432e-bec5-c3c50e60b81a\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.574753 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.574640 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:30.737311 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.737254 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh"] Jul 09 22:53:30.740033 ip-10-0-135-244 kubenswrapper[2576]: W0709 22:53:30.740005 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod75c8fc9a_fe24_432e_bec5_c3c50e60b81a.slice/crio-59491a538fd19929c10f921717cb92fd9f6f839df9e51ec33292dafdc01c8d9e WatchSource:0}: Error finding container 59491a538fd19929c10f921717cb92fd9f6f839df9e51ec33292dafdc01c8d9e: Status 404 returned error can't find the container with id 59491a538fd19929c10f921717cb92fd9f6f839df9e51ec33292dafdc01c8d9e Jul 09 22:53:30.795987 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.795966 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/1.log" Jul 09 22:53:30.801206 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:30.801191 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-nr6vw_e8d556ba-1556-4990-a7fa-b6bd1399ed5b/console-operator/2.log" Jul 09 22:53:31.213784 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:31.213688 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" event={"ID":"75c8fc9a-fe24-432e-bec5-c3c50e60b81a","Type":"ContainerStarted","Data":"069d95ba499f6fa02da729674143cca716d1a5f9fb3b0a8baad4a9ea3b296324"} Jul 09 22:53:31.213784 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:31.213747 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" event={"ID":"75c8fc9a-fe24-432e-bec5-c3c50e60b81a","Type":"ContainerStarted","Data":"59491a538fd19929c10f921717cb92fd9f6f839df9e51ec33292dafdc01c8d9e"} Jul 09 22:53:31.213985 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:31.213834 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:31.239014 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:31.238968 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" podStartSLOduration=1.238956072 podStartE2EDuration="1.238956072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:53:31.237541997 +0000 UTC m=+2679.335757186" watchObservedRunningTime="2026-07-09 22:53:31.238956072 +0000 UTC m=+2679.337171261" Jul 09 22:53:33.225671 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:33.225642 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dcgsk_6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c/dns/0.log" Jul 09 22:53:33.256742 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:33.256688 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dcgsk_6b605cb1-c348-4a4a-bbd6-bdfb4b34bb1c/kube-rbac-proxy/0.log" Jul 09 22:53:33.380708 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:33.380671 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-65pnm_dff8d8d0-93d0-40a2-8061-9fb8f8903316/dns-node-resolver/0.log" Jul 09 22:53:33.993124 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:33.993094 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-c5d578676-dfpfv_d6c84071-071a-4247-a8c1-00db7051e14a/registry/0.log" Jul 09 22:53:34.092887 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:34.092860 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lj48t_7e53afda-97f8-4484-9c7c-b747bde49e8f/node-ca/0.log" Jul 09 22:53:35.440530 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:35.440501 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ff5d987cc-rf44x_c7605dd3-5b80-465f-912f-19b91742c54b/kube-auth-proxy/0.log" Jul 09 22:53:35.495421 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:35.495391 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thsbt_92b29da5-7df6-408f-975f-bba71829b164/istio-proxy/0.log" Jul 09 22:53:35.525094 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:35.525064 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thsbt_92b29da5-7df6-408f-975f-bba71829b164/istio-proxy/1.log" Jul 09 22:53:35.568896 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:35.568874 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jckm7_ffe555a7-1594-4cfa-8a16-687d9f46afad/payload-pre-processing/0.log" Jul 09 22:53:35.620416 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:35.620388 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qgvd2_316a095c-7dd4-442f-ad2c-19edb5b7fb09/payload-processing/0.log" Jul 09 22:53:35.649786 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:35.649768 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58c6fd45c7-x2b2x_e28edf89-6b16-4538-97f4-7f72e368a48d/router/0.log" Jul 09 22:53:36.520579 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:36.520540 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rxhvj_e252f3c5-1f99-4dd1-8d97-8d666602d58c/serve-healthcheck-canary/0.log" Jul 09 22:53:37.229120 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:37.229094 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-sh4fh" Jul 09 22:53:37.281232 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:37.281205 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmjsh_c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f/kube-rbac-proxy/0.log" Jul 09 22:53:37.315321 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:37.315296 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmjsh_c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f/exporter/0.log" Jul 09 22:53:37.344517 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:37.344496 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kmjsh_c5ba8067-5c36-4bb1-9cb0-33ab88d95b1f/extractor/0.log" Jul 09 22:53:39.816944 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:39.816864 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-chzl4_fb70b5d3-8bde-4df1-90f4-fee384cdad1e/manager/0.log" Jul 09 22:53:39.958803 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:39.958769 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xpd77_59df5409-675c-4c7f-9f90-cd474d423f75/manager/0.log" Jul 09 22:53:40.140161 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:40.140129 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-22mwk_9aa9a7d2-8530-4b4b-b0f4-35828b196c89/server/0.log" Jul 09 22:53:40.212027 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:40.212001 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b5p8s_4481c981-a256-41e3-a1dc-23eddfe664a4/manager/0.log" Jul 09 22:53:40.453925 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:40.453899 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b79867d65-kt8xm_a0fdc908-ab18-4b60-9d33-8e272264b19e/manager/0.log" Jul 09 22:53:40.485139 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:40.485106 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b79867d65-kt8xm_a0fdc908-ab18-4b60-9d33-8e272264b19e/copy-manifests/0.log" Jul 09 22:53:41.333243 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:41.333213 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ff5d987cc-rf44x_c7605dd3-5b80-465f-912f-19b91742c54b/kube-auth-proxy/0.log" Jul 09 22:53:41.366235 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:41.366207 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thsbt_92b29da5-7df6-408f-975f-bba71829b164/istio-proxy/0.log" Jul 09 22:53:41.396546 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:41.396511 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thsbt_92b29da5-7df6-408f-975f-bba71829b164/istio-proxy/1.log" Jul 09 22:53:41.419097 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:41.419061 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jckm7_ffe555a7-1594-4cfa-8a16-687d9f46afad/payload-pre-processing/0.log" Jul 09 22:53:41.442191 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:41.442163 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qgvd2_316a095c-7dd4-442f-ad2c-19edb5b7fb09/payload-processing/0.log" Jul 09 22:53:41.457000 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:41.456979 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58c6fd45c7-x2b2x_e28edf89-6b16-4538-97f4-7f72e368a48d/router/0.log" Jul 09 22:53:42.964003 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:42.963968 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ff5d987cc-rf44x_c7605dd3-5b80-465f-912f-19b91742c54b/kube-auth-proxy/0.log" Jul 09 22:53:42.992081 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:42.992048 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thsbt_92b29da5-7df6-408f-975f-bba71829b164/istio-proxy/0.log" Jul 09 22:53:43.021001 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:43.020965 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-thsbt_92b29da5-7df6-408f-975f-bba71829b164/istio-proxy/1.log" Jul 09 22:53:43.049670 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:43.049636 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-jckm7_ffe555a7-1594-4cfa-8a16-687d9f46afad/payload-pre-processing/0.log" Jul 09 22:53:43.073732 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:43.073693 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qgvd2_316a095c-7dd4-442f-ad2c-19edb5b7fb09/payload-processing/0.log" Jul 09 22:53:43.093996 ip-10-0-135-244 kubenswrapper[2576]: I0709 22:53:43.093955 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58c6fd45c7-x2b2x_e28edf89-6b16-4538-97f4-7f72e368a48d/router/0.log"