Jul 15 20:57:10.454932 ip-10-0-135-80 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 15 20:57:10.454944 ip-10-0-135-80 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 15 20:57:10.454953 ip-10-0-135-80 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 15 20:57:10.455274 ip-10-0-135-80 systemd[1]: Failed to start Kubernetes Kubelet. Jul 15 20:57:19.791437 ip-10-0-135-80 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 15 20:57:19.791452 ip-10-0-135-80 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3e12f6e421894e39b2b4f45032e59753 -- Jul 15 20:59:06.789448 ip-10-0-135-80 systemd[1]: Starting Kubernetes Kubelet... Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.204281 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207833 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207845 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207849 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207853 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207857 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207860 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207863 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 20:59:07.239632 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207866 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207873 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207876 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207879 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207882 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207885 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207888 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207891 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207894 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207899 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207902 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207905 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207908 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207911 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207914 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207917 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207920 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207923 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207926 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 20:59:07.240553 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207928 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207932 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207934 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207938 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207941 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207944 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207946 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207949 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207951 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207955 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207957 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207960 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207964 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207967 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207969 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207975 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207978 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207981 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207983 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207986 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207988 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 20:59:07.241087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207991 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207994 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.207997 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208000 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208002 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208004 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208007 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208009 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208012 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208014 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208017 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208019 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208022 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208024 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208027 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208029 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208032 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208034 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208037 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208040 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 20:59:07.241782 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208042 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208045 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208048 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208050 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208053 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208056 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208058 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208062 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208065 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208067 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208070 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208072 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208075 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208078 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208080 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208083 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208085 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208088 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208090 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 20:59:07.242413 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208094 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208098 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208100 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208105 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208108 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208112 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208114 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208117 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208119 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208122 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208128 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208130 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208133 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208873 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208880 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208883 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208886 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208888 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208891 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 20:59:07.242960 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208894 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208897 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208902 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208905 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208908 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208911 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208913 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208916 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208919 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208921 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208924 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208927 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208931 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208934 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208937 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208940 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208943 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208946 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208949 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 20:59:07.243718 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208951 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208954 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208957 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208959 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208964 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208967 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208970 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208973 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208975 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208978 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208981 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208984 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208986 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208989 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208992 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208994 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.208997 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209000 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209004 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 20:59:07.244360 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209007 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209012 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209017 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209019 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209022 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209025 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209028 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209030 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209034 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209037 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209040 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209043 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209045 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209048 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209051 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209053 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209056 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209060 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209063 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 20:59:07.245114 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209065 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209068 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209071 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209073 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209075 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209078 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209080 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209083 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209086 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209088 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209091 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209094 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209097 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209099 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209101 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209104 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209106 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209109 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209111 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209113 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 20:59:07.245650 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209116 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209118 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209120 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209123 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209126 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209129 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209131 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209134 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209136 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209139 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209143 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209146 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209148 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209151 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209154 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209156 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209573 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209579 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209583 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209587 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 20:59:07.246235 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209590 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209593 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209596 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209600 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209603 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209605 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209608 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209611 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209613 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209616 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209619 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209621 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209623 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209626 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209628 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209631 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209633 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209636 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209639 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209642 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 20:59:07.246819 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209644 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209647 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209653 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209655 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209658 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209660 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209663 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209665 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209668 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209671 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209673 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209676 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209679 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209681 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209684 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209686 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209689 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209692 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209695 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209697 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 20:59:07.247441 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209705 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209707 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209709 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209713 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209717 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209720 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209723 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209726 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209728 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209731 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209733 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209736 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209738 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209741 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209746 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209748 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209751 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209753 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209756 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 20:59:07.247963 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209759 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209762 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209764 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209768 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209772 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209774 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209777 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209780 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209782 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209785 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209788 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209790 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209793 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209798 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209800 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209802 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209805 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209807 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209810 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 20:59:07.248510 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209812 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209816 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209818 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209821 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209823 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209826 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209828 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209830 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209835 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209837 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209840 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209843 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209845 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209847 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209850 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209853 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.209855 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209933 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209948 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209957 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209964 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 20:59:07.249323 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209969 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209972 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209976 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209981 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209985 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209990 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209993 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209996 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.209999 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210002 2573 flags.go:64] FLAG: --cgroup-root="" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210005 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210008 2573 flags.go:64] FLAG: --client-ca-file="" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210011 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210013 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210017 2573 flags.go:64] FLAG: --cluster-domain="" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210021 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210024 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210027 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210031 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210036 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210039 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210042 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210045 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210048 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 15 20:59:07.249952 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210051 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210054 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210057 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210060 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210064 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210067 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210070 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210072 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210075 2573 flags.go:64] FLAG: --enable-server="true" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210078 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210083 2573 flags.go:64] FLAG: --event-burst="100" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210086 2573 flags.go:64] FLAG: --event-qps="50" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210090 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210094 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210097 2573 flags.go:64] FLAG: --eviction-hard="" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210101 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210104 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210107 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210109 2573 flags.go:64] FLAG: --eviction-soft="" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210112 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210115 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210118 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210121 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210123 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210126 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 20:59:07.390236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210129 2573 flags.go:64] FLAG: --feature-gates="" Jul 15 20:59:07.279758 ip-10-0-135-80 systemd[1]: Started Kubernetes Kubelet. Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210135 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210139 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210142 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210145 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210148 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210151 2573 flags.go:64] FLAG: --help="false" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210154 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210157 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210160 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210163 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210185 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210189 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210192 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210194 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210197 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210200 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210203 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210206 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210209 2573 flags.go:64] FLAG: --kube-reserved="" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210214 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210217 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210220 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210222 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 20:59:07.406327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210225 2573 flags.go:64] FLAG: --lock-file="" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210228 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210231 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210234 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210239 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210242 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210245 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210248 2573 flags.go:64] FLAG: --logging-format="text" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210251 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210254 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210258 2573 flags.go:64] FLAG: --manifest-url="" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210261 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210265 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210268 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210272 2573 flags.go:64] FLAG: --max-pods="110" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210275 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210278 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210281 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210284 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210287 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210290 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210293 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210300 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210303 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210306 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 20:59:07.407254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210309 2573 flags.go:64] FLAG: --pod-cidr="" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210312 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210318 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210323 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210326 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210329 2573 flags.go:64] FLAG: --port="10250" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210332 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210335 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0da777ed781692a3b" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210338 2573 flags.go:64] FLAG: --qos-reserved="" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210341 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210344 2573 flags.go:64] FLAG: --register-node="true" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210347 2573 flags.go:64] FLAG: --register-with-taints="" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210350 2573 flags.go:64] FLAG: --registry-burst="10" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210353 2573 flags.go:64] FLAG: --registry-qps="5" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210356 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210359 2573 flags.go:64] FLAG: --reserved-memory="" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210362 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210365 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210370 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210373 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210376 2573 flags.go:64] FLAG: --runonce="false" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210379 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210382 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210385 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210388 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 20:59:07.408152 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210390 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210393 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210396 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210399 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210402 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210405 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210408 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210411 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210414 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210416 2573 flags.go:64] FLAG: --system-cgroups="" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210421 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210426 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210429 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210432 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210436 2573 flags.go:64] FLAG: --tls-min-version="" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210438 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210441 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210444 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210447 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210449 2573 flags.go:64] FLAG: --v="2" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210453 2573 flags.go:64] FLAG: --version="false" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210457 2573 flags.go:64] FLAG: --vmodule="" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210461 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.210464 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210582 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 20:59:07.409010 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210587 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210590 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210593 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210595 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210598 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210601 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210603 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210606 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210609 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210611 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210614 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210617 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210619 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210622 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210624 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210627 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210629 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210633 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210636 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210639 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 20:59:07.409790 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210641 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210644 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210648 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210652 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210655 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210658 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210661 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210664 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210666 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210669 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210672 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210675 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210679 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210682 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210685 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210687 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210690 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210692 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210695 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210699 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 20:59:07.410411 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210702 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210704 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210707 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210709 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210712 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210715 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210718 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210720 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210757 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210762 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210779 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210783 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210786 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210984 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210986 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210989 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.210992 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211214 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211400 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211403 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 20:59:07.411063 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211406 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211409 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211607 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211610 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211614 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211618 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211816 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211820 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.211823 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212019 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212022 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212025 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212028 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212030 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212237 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212433 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212436 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212440 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212442 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 20:59:07.411702 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212445 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212641 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212646 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212648 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212651 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212847 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212850 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212854 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212856 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212860 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212863 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212866 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212870 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212874 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212876 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212879 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212882 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212887 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 20:59:07.412320 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.212889 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.213883 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.226686 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.226702 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.227344 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.227352 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227410 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227416 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227420 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227423 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227426 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227429 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227433 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227436 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227439 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 20:59:07.412851 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227441 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227444 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227447 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227449 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227452 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227455 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227458 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227460 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227463 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227465 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227468 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227470 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227473 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227476 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227479 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227482 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227484 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227488 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227490 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227493 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 20:59:07.413396 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227497 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227501 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227504 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227507 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227510 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227512 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227515 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227518 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227521 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227523 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227525 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227528 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227531 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227533 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227536 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227539 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227541 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227545 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227548 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 20:59:07.413970 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227550 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227553 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227557 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227560 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227563 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227566 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227569 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227572 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227577 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227579 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227582 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227585 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227588 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227591 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227594 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227597 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227600 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227602 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227605 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 20:59:07.414540 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227609 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227612 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227616 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227618 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227621 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227624 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227627 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227630 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227633 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227636 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227639 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227641 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227644 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227647 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227650 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227652 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227654 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227657 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227660 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227662 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 20:59:07.415473 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227665 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227668 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227671 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227673 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227676 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227678 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227681 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227683 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227686 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227689 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227691 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227694 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.227699 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227837 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227844 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 20:59:07.416355 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227850 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227853 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227856 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227859 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227862 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227865 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227868 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227871 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227874 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227876 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227879 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227882 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227885 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227888 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227890 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227893 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227895 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227898 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227902 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227904 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 20:59:07.417015 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227907 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227909 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227912 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227915 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227917 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227920 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227922 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227925 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227928 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227930 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227933 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227935 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227940 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227942 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227945 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227947 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227950 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227953 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227955 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227958 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 20:59:07.418101 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227960 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227963 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227965 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227968 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227971 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227973 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227976 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227978 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227981 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227983 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227987 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227990 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227992 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227995 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.227997 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228000 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228003 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228005 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228007 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228010 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 20:59:07.418983 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228013 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228015 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228017 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228020 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228024 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228026 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228029 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228031 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228034 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228036 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228040 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228042 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228045 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228047 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228050 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228052 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228055 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228058 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228060 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 20:59:07.419855 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228063 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228067 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228070 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228074 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228077 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228080 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228082 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228086 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228089 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228091 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228094 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228097 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228099 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228103 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228107 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228110 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228112 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 20:59:07.422616 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:07.228117 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.228122 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.228253 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.231061 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.232370 2573 server.go:1019] "Starting client certificate rotation" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.232466 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.232500 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.235254 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.236870 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.245703 2573 log.go:25] "Validated CRI v1 runtime API" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.248358 2573 log.go:25] "Validated CRI v1 image API" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.249222 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 20:59:07.423487 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.252722 2573 fs.go:135] Filesystem UUIDs: map[628845f4-2ee8-455d-9e1e-56a227a64d94:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d9da5280-00f4-467f-8b88-fff69ddeccf8:/dev/nvme0n1p4] Jul 15 20:59:07.424768 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.252736 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 20:59:07.424975 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.261657 2573 manager.go:216] Machine: {Timestamp:2026-07-15 20:59:07.26021027 +0000 UTC m=+0.350443440 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101882 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21a9234757e0466b7ad4b065ab516a SystemUUID:ec21a923-4757-e046-6b7a-d4b065ab516a BootID:3e12f6e4-2189-4e39-b2b4-f45032e59753 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c5:43:49:df:7f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c5:43:49:df:7f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:27:75:8f:73:c9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 20:59:07.424975 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.261758 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 20:59:07.424975 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.261873 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 20:59:07.424975 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.263086 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 20:59:07.424975 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.263111 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-80.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 20:59:07.424975 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.263938 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 20:59:07.424975 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.263948 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 20:59:07.424975 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.263956 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.264599 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.264695 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.265962 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.268936 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.269069 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.269100 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.269113 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.269126 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.269138 2573 kubelet.go:398] "Adding apiserver pod source" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.269152 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.269529 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.269548 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.271862 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.273720 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.273748 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.273893 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.273905 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.273911 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.273916 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.273925 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.273930 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.274578 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.274585 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.274591 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.274597 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.274605 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.274613 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.276002 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.276010 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.278896 2573 server.go:1268] "Started kubelet" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.278969 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.279043 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.279068 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.280241 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.281509 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.282622 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-80.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.282628 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.284150 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 20:59:07.608343 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.287404 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.288370 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.288378 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.288434 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.289048 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.289049 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.289076 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.289200 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.289211 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.289321 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.289807 2573 factory.go:55] Registering systemd factory Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.289815 2573 factory.go:223] Registration of the systemd container factory successfully Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.289989 2573 factory.go:153] Registering CRI-O factory Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.290000 2573 factory.go:223] Registration of the crio container factory successfully Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.290022 2573 factory.go:103] Registering Raw factory Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.290035 2573 manager.go:1196] Started watching for new ooms in manager Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.290865 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.290891 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.291518 2573 manager.go:349] Starting recovery of all containers Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.291213 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-80.ec2.internal.18c2923255df8dbf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-80.ec2.internal,UID:ip-10-0-135-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-80.ec2.internal,},FirstTimestamp:2026-07-15 20:59:07.278871999 +0000 UTC m=+0.369105168,LastTimestamp:2026-07-15 20:59:07.278871999 +0000 UTC m=+0.369105168,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-80.ec2.internal,}" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.302914 2573 manager.go:354] Recovery completed Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.307094 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.308942 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientMemory" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.308968 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.308978 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientPID" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.309557 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.309563 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.309570 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.309587 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.310224 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientMemory" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.310246 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.310256 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientPID" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.310912 2573 policy_none.go:49] "None policy: Start" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.310924 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 20:59:07.610908 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.310945 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.311205 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-80.ec2.internal.18c2923257aa9934 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-80.ec2.internal,UID:ip-10-0-135-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-80.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-80.ec2.internal,},FirstTimestamp:2026-07-15 20:59:07.308955956 +0000 UTC m=+0.399189128,LastTimestamp:2026-07-15 20:59:07.308955956 +0000 UTC m=+0.399189128,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-80.ec2.internal,}" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.311974 2573 policy_none.go:47] "Start" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.314919 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pxcbw" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.325080 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-80.ec2.internal.18c2923257aadd47 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-80.ec2.internal,UID:ip-10-0-135-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-80.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-80.ec2.internal,},FirstTimestamp:2026-07-15 20:59:07.308973383 +0000 UTC m=+0.399206552,LastTimestamp:2026-07-15 20:59:07.308973383 +0000 UTC m=+0.399206552,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-80.ec2.internal,}" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.326239 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pxcbw" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.352254 2573 manager.go:337] "Starting Device Plugin manager" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.352422 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.352432 2573 server.go:87] "Starting device plugin registration server" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.352626 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.352635 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.352767 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.352853 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.352862 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.353296 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.353335 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.439291 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.440501 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.440536 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.440563 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.440596 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.443672 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.453593 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.454557 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientMemory" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.454594 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.454608 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientPID" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.454640 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.616512 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.467621 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.467642 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-80.ec2.internal\": node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.497683 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.542940 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal"] Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.543024 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.544682 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientMemory" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.544710 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.544723 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientPID" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.548981 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.549098 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.549127 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.549691 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientMemory" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.549697 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientMemory" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.549718 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.549730 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientPID" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.549719 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.549819 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientPID" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.551558 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.551590 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.552298 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientMemory" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.552315 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.552324 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientPID" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.578561 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-80.ec2.internal\" not found" node="ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.580151 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-80.ec2.internal\" not found" node="ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.591768 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d8e486c8c5ef440afb48d0d5efafcdc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal\" (UID: \"2d8e486c8c5ef440afb48d0d5efafcdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.592422 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d8e486c8c5ef440afb48d0d5efafcdc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal\" (UID: \"2d8e486c8c5ef440afb48d0d5efafcdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.592869 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3eb8d182e636c274e1ac9b37ef4dec65-config\") pod \"kube-apiserver-proxy-ip-10-0-135-80.ec2.internal\" (UID: \"3eb8d182e636c274e1ac9b37ef4dec65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.617927 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.598011 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:07.814795 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.693964 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d8e486c8c5ef440afb48d0d5efafcdc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal\" (UID: \"2d8e486c8c5ef440afb48d0d5efafcdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.814795 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.694008 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d8e486c8c5ef440afb48d0d5efafcdc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal\" (UID: \"2d8e486c8c5ef440afb48d0d5efafcdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.814795 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.694025 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3eb8d182e636c274e1ac9b37ef4dec65-config\") pod \"kube-apiserver-proxy-ip-10-0-135-80.ec2.internal\" (UID: \"3eb8d182e636c274e1ac9b37ef4dec65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.814795 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.694096 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3eb8d182e636c274e1ac9b37ef4dec65-config\") pod \"kube-apiserver-proxy-ip-10-0-135-80.ec2.internal\" (UID: \"3eb8d182e636c274e1ac9b37ef4dec65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.814795 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.694148 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d8e486c8c5ef440afb48d0d5efafcdc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal\" (UID: \"2d8e486c8c5ef440afb48d0d5efafcdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.814795 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.694166 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d8e486c8c5ef440afb48d0d5efafcdc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal\" (UID: \"2d8e486c8c5ef440afb48d0d5efafcdc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.814795 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.699190 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:07.814795 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.800393 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:07.988685 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.883278 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.988685 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:07.883291 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal" Jul 15 20:59:07.988685 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:07.901216 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:08.176102 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:08.008747 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:08.176102 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:08.109472 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:08.385271 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:08.210460 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:08.385271 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.232704 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 20:59:08.385271 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.232838 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 20:59:08.385271 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:08.310619 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:08.385271 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.329854 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 20:54:07 +0000 UTC" deadline="2028-03-15 19:36:59.588728628 +0000 UTC" Jul 15 20:59:08.385271 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.329909 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14614h37m51.258823983s" Jul 15 20:59:08.575556 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:08.411667 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:08.575556 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:08.496049 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb8d182e636c274e1ac9b37ef4dec65.slice/crio-4ef28425133d65dc7d9cc9578d0217e92c3c3ceda83ab723ab73eedcb5ce3ac1 WatchSource:0}: Error finding container 4ef28425133d65dc7d9cc9578d0217e92c3c3ceda83ab723ab73eedcb5ce3ac1: Status 404 returned error can't find the container with id 4ef28425133d65dc7d9cc9578d0217e92c3c3ceda83ab723ab73eedcb5ce3ac1 Jul 15 20:59:08.575556 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.500862 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 20:59:08.575556 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.509288 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 20:59:08.575556 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.510191 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientMemory" Jul 15 20:59:08.575556 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.510222 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 20:59:08.575556 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.510265 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeHasSufficientPID" Jul 15 20:59:08.575556 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:08.512434 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:08.575556 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.540723 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 20:59:08.816362 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:08.612582 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:08.816362 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.637359 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 20:59:08.816362 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:08.713406 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:08.816362 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:08.813584 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-80.ec2.internal\" not found" Jul 15 20:59:09.017124 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.869008 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 20:59:09.017124 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.888695 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal" Jul 15 20:59:09.017124 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.901270 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 20:59:09.017124 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.902597 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" Jul 15 20:59:09.017124 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:08.916328 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 20:59:09.274620 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.274592 2573 apiserver.go:52] "Watching apiserver" Jul 15 20:59:09.285315 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.284258 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 20:59:09.285315 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.284831 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-zjhlw","openshift-dns/node-resolver-pw8ln","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal","openshift-multus/multus-additional-cni-plugins-zpdrq","openshift-multus/multus-j9sfn","openshift-multus/network-metrics-daemon-kgvtw","openshift-network-diagnostics/network-check-target-z7dvf","openshift-network-operator/iptables-alerter-jf7j4","kube-system/global-pull-secret-syncer-98h25","kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh","openshift-cluster-node-tuning-operator/tuned-9cgzq","openshift-image-registry/node-ca-6pkk7","openshift-ovn-kubernetes/ovnkube-node-v2zh8"] Jul 15 20:59:09.287096 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.287075 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.289076 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.289055 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 20:59:09.289808 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.289563 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:09.290962 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.290944 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:09.291052 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.291031 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:09.292367 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.291736 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 20:59:09.292367 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.291997 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 20:59:09.292367 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.292172 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 20:59:09.292367 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.292344 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-clflm\"" Jul 15 20:59:09.293193 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.292818 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 20:59:09.293193 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.292892 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-x66tw\"" Jul 15 20:59:09.293193 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.293048 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 20:59:09.293431 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.293216 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 20:59:09.293431 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.293285 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.295459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.295127 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:09.295459 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.295217 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:09.296379 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.296170 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kkqcg\"" Jul 15 20:59:09.296379 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.296222 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 20:59:09.296379 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.296222 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 20:59:09.297683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.297282 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.299516 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.299262 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.299932 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.299867 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lw46w\"" Jul 15 20:59:09.300320 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.300131 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 20:59:09.300320 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.300226 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 20:59:09.301388 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.301361 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:09.301484 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.301430 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:09.302737 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.302521 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:59:09.302737 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.302576 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 20:59:09.304277 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.303557 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 20:59:09.304277 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.303568 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rz4vf\"" Jul 15 20:59:09.304277 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.303949 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.304484 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.304461 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 20:59:09.304536 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.304466 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:09.304590 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.304572 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-system-cni-dir\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.304646 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.304616 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-cnibin\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.304699 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.304645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd98705f-b772-418d-ba13-383f7fcda593-cni-binary-copy\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.304699 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.304683 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-multus-socket-dir-parent\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.304827 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.304721 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-run-k8s-cni-cncf-io\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.304886 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.304867 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-run-netns\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305009 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.304994 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-var-lib-cni-bin\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305217 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305094 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-multus-cni-dir\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305217 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305138 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-var-lib-kubelet\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305217 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305172 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dd98705f-b772-418d-ba13-383f7fcda593-multus-daemon-config\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305416 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-etc-kubernetes\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305416 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305296 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgt8p\" (UniqueName: \"kubernetes.io/projected/dd98705f-b772-418d-ba13-383f7fcda593-kube-api-access-qgt8p\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305416 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305340 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de9654f9-9400-474b-8978-582043f3d11e-agent-certs\") pod \"konnectivity-agent-zjhlw\" (UID: \"de9654f9-9400-474b-8978-582043f3d11e\") " pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:09.305416 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305387 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-hostroot\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305628 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305454 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de9654f9-9400-474b-8978-582043f3d11e-konnectivity-ca\") pod \"konnectivity-agent-zjhlw\" (UID: \"de9654f9-9400-474b-8978-582043f3d11e\") " pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:09.305628 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305488 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-os-release\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305628 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305513 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-var-lib-cni-multus\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305628 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305544 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-multus-conf-dir\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.305628 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.305590 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-run-multus-certs\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.307032 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.307010 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.307325 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.307212 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 20:59:09.307669 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.307646 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 20:59:09.307756 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.307747 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 20:59:09.309259 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.309032 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nbswk\"" Jul 15 20:59:09.309259 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.309177 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.310083 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.309932 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tcpzq\"" Jul 15 20:59:09.310175 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.310126 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:59:09.311995 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.310645 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 20:59:09.313104 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.312491 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.313902 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.313606 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 20:59:09.313902 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.313758 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f8hdd\"" Jul 15 20:59:09.313902 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.313799 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 20:59:09.313902 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.313846 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 20:59:09.315552 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.315317 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 20:59:09.316303 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.315914 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 20:59:09.316303 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.315947 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dzc94\"" Jul 15 20:59:09.316303 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.316122 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 20:59:09.316303 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.316200 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 20:59:09.316467 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.316346 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 20:59:09.318401 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.317080 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 20:59:09.329960 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.329939 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9jwmr" Jul 15 20:59:09.337656 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.337347 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9jwmr" Jul 15 20:59:09.390296 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.390277 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 20:59:09.406733 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406709 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-system-cni-dir\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.406857 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406747 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-var-lib-cni-bin\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.406857 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de9654f9-9400-474b-8978-582043f3d11e-agent-certs\") pod \"konnectivity-agent-zjhlw\" (UID: \"de9654f9-9400-474b-8978-582043f3d11e\") " pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:09.406857 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406804 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.407027 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406845 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-var-lib-cni-bin\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407027 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406860 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-system-cni-dir\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407027 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-systemd\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.407027 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406910 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-run-netns\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.407027 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406950 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-cni-netd\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.407027 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.406980 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.407027 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407007 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-os-release\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407034 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-multus-conf-dir\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407056 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-run-multus-certs\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-kubelet\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407125 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-registration-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407132 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-os-release\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407149 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-cnibin\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407213 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-run-multus-certs\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407223 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz4f8\" (UniqueName: \"kubernetes.io/projected/17f76060-64b0-4ec1-ae6e-18eaea915761-kube-api-access-nz4f8\") pod \"node-resolver-pw8ln\" (UID: \"17f76060-64b0-4ec1-ae6e-18eaea915761\") " pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407257 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-run\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407261 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-multus-conf-dir\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407260 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407284 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-lib-modules\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407335 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-multus-cni-dir\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-var-lib-kubelet\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.407417 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-slash\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-run-systemd\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407442 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bk9k\" (UniqueName: \"kubernetes.io/projected/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-kube-api-access-4bk9k\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407467 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-env-overrides\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17f76060-64b0-4ec1-ae6e-18eaea915761-hosts-file\") pod \"node-resolver-pw8ln\" (UID: \"17f76060-64b0-4ec1-ae6e-18eaea915761\") " pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407517 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-var-lib-kubelet\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407530 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-socket-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407560 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407573 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-multus-cni-dir\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407581 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-sys\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407628 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjbs7\" (UniqueName: \"kubernetes.io/projected/fda8f067-5b21-4d25-9771-77f72ab05e60-kube-api-access-rjbs7\") pod \"node-ca-6pkk7\" (UID: \"fda8f067-5b21-4d25-9771-77f72ab05e60\") " pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407676 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-var-lib-openvswitch\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407710 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-ovnkube-config\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407733 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:09.408112 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407770 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01016934-00c3-4dae-8db6-a118e7c3cbe9-dbus\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407802 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407822 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-cni-binary-copy\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407841 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-tuned\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407864 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fda8f067-5b21-4d25-9771-77f72ab05e60-serviceca\") pod \"node-ca-6pkk7\" (UID: \"fda8f067-5b21-4d25-9771-77f72ab05e60\") " pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407899 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-cnibin\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407932 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd98705f-b772-418d-ba13-383f7fcda593-cni-binary-copy\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407959 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-multus-socket-dir-parent\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407983 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-cnibin\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.407986 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-node-log\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408018 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2cg9\" (UniqueName: \"kubernetes.io/projected/b0c35751-fbd2-4792-987a-fa92ce47b882-kube-api-access-w2cg9\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408044 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-sysconfig\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408057 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-multus-socket-dir-parent\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408074 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dd98705f-b772-418d-ba13-383f7fcda593-multus-daemon-config\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.409012 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408319 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-run-openvswitch\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408351 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-run-ovn\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408372 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4vvz\" (UniqueName: \"kubernetes.io/projected/3ebd2686-2acc-4181-9351-9fadd5d6d6a4-kube-api-access-d4vvz\") pod \"iptables-alerter-jf7j4\" (UID: \"3ebd2686-2acc-4181-9351-9fadd5d6d6a4\") " pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408392 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408413 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-sys-fs\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408443 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-host\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408466 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de9654f9-9400-474b-8978-582043f3d11e-konnectivity-ca\") pod \"konnectivity-agent-zjhlw\" (UID: \"de9654f9-9400-474b-8978-582043f3d11e\") " pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408486 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-ovn-node-metrics-cert\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408506 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17f76060-64b0-4ec1-ae6e-18eaea915761-tmp-dir\") pod \"node-resolver-pw8ln\" (UID: \"17f76060-64b0-4ec1-ae6e-18eaea915761\") " pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408528 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ebd2686-2acc-4181-9351-9fadd5d6d6a4-iptables-alerter-script\") pod \"iptables-alerter-jf7j4\" (UID: \"3ebd2686-2acc-4181-9351-9fadd5d6d6a4\") " pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-etc-selinux\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408577 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-os-release\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408602 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408630 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fda8f067-5b21-4d25-9771-77f72ab05e60-host\") pod \"node-ca-6pkk7\" (UID: \"fda8f067-5b21-4d25-9771-77f72ab05e60\") " pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.409617 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408669 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd98705f-b772-418d-ba13-383f7fcda593-cni-binary-copy\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408680 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-var-lib-cni-multus\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408713 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-cni-bin\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408739 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-modprobe-d\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408764 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dd98705f-b772-418d-ba13-383f7fcda593-multus-daemon-config\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408768 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01016934-00c3-4dae-8db6-a118e7c3cbe9-kubelet-config\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408842 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408849 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-var-lib-cni-multus\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408872 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-run-k8s-cni-cncf-io\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408899 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-run-netns\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408924 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-etc-kubernetes\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408963 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-log-socket\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.408992 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ebd2686-2acc-4181-9351-9fadd5d6d6a4-host-slash\") pod \"iptables-alerter-jf7j4\" (UID: \"3ebd2686-2acc-4181-9351-9fadd5d6d6a4\") " pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409024 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-run-netns\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409024 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-etc-kubernetes\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409063 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-device-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409085 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-host-run-k8s-cni-cncf-io\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409119 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgt8p\" (UniqueName: \"kubernetes.io/projected/dd98705f-b772-418d-ba13-383f7fcda593-kube-api-access-qgt8p\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409138 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-systemd-units\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409195 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-ovnkube-script-lib\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtg8g\" (UniqueName: \"kubernetes.io/projected/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-kube-api-access-vtg8g\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409263 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-kubernetes\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409284 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-sysctl-d\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409313 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-sysctl-conf\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409334 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-var-lib-kubelet\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409359 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-hostroot\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409383 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409410 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dd98705f-b772-418d-ba13-383f7fcda593-hostroot\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.410724 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409389 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de9654f9-9400-474b-8978-582043f3d11e-konnectivity-ca\") pod \"konnectivity-agent-zjhlw\" (UID: \"de9654f9-9400-474b-8978-582043f3d11e\") " pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:09.411443 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409412 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npjhj\" (UniqueName: \"kubernetes.io/projected/8512fe14-5aac-4bfb-9e84-e62f0997c20c-kube-api-access-npjhj\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.411443 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409471 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-etc-openvswitch\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.411443 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409506 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4bm7\" (UniqueName: \"kubernetes.io/projected/343ee524-4530-4307-b119-1c4fb610db6f-kube-api-access-j4bm7\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:09.411443 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-system-cni-dir\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.411443 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.409980 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8512fe14-5aac-4bfb-9e84-e62f0997c20c-tmp\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.411745 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.411703 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de9654f9-9400-474b-8978-582043f3d11e-agent-certs\") pod \"konnectivity-agent-zjhlw\" (UID: \"de9654f9-9400-474b-8978-582043f3d11e\") " pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:09.423445 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.423417 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 20:59:09.423535 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.423452 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 20:59:09.423535 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.423468 2573 projected.go:196] Error preparing data for projected volume kube-api-access-gv5fg for pod openshift-network-diagnostics/network-check-target-z7dvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:09.423605 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.423540 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg podName:f37e3221-e7d4-486e-8f9c-5c0d83e702dd nodeName:}" failed. No retries permitted until 2026-07-15 20:59:09.923518494 +0000 UTC m=+3.013751655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gv5fg" (UniqueName: "kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg") pod "network-check-target-z7dvf" (UID: "f37e3221-e7d4-486e-8f9c-5c0d83e702dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:09.426069 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.426047 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgt8p\" (UniqueName: \"kubernetes.io/projected/dd98705f-b772-418d-ba13-383f7fcda593-kube-api-access-qgt8p\") pod \"multus-j9sfn\" (UID: \"dd98705f-b772-418d-ba13-383f7fcda593\") " pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.449661 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.449599 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal" event={"ID":"3eb8d182e636c274e1ac9b37ef4dec65","Type":"ContainerStarted","Data":"4ef28425133d65dc7d9cc9578d0217e92c3c3ceda83ab723ab73eedcb5ce3ac1"} Jul 15 20:59:09.451082 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.451022 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" event={"ID":"2d8e486c8c5ef440afb48d0d5efafcdc","Type":"ContainerStarted","Data":"9cd9081f0b62494be637ab34293919b99010c4850f64c237dbd215fe49fbc742"} Jul 15 20:59:09.510511 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510478 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-run\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.510683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510521 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-lib-modules\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.510683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510548 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-slash\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.510683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-run-systemd\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.510683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510595 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bk9k\" (UniqueName: \"kubernetes.io/projected/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-kube-api-access-4bk9k\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.510683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510620 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-env-overrides\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.510683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510630 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-slash\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510692 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-run-systemd\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510704 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-lib-modules\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17f76060-64b0-4ec1-ae6e-18eaea915761-hosts-file\") pod \"node-resolver-pw8ln\" (UID: \"17f76060-64b0-4ec1-ae6e-18eaea915761\") " pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510754 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17f76060-64b0-4ec1-ae6e-18eaea915761-hosts-file\") pod \"node-resolver-pw8ln\" (UID: \"17f76060-64b0-4ec1-ae6e-18eaea915761\") " pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510766 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-socket-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510826 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-sys\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510852 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjbs7\" (UniqueName: \"kubernetes.io/projected/fda8f067-5b21-4d25-9771-77f72ab05e60-kube-api-access-rjbs7\") pod \"node-ca-6pkk7\" (UID: \"fda8f067-5b21-4d25-9771-77f72ab05e60\") " pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510880 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-var-lib-openvswitch\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-ovnkube-config\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510924 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-sys\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510934 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510960 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01016934-00c3-4dae-8db6-a118e7c3cbe9-dbus\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510967 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-var-lib-openvswitch\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.511003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510986 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511012 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-cni-binary-copy\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511038 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-tuned\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fda8f067-5b21-4d25-9771-77f72ab05e60-serviceca\") pod \"node-ca-6pkk7\" (UID: \"fda8f067-5b21-4d25-9771-77f72ab05e60\") " pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511092 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-node-log\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511121 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2cg9\" (UniqueName: \"kubernetes.io/projected/b0c35751-fbd2-4792-987a-fa92ce47b882-kube-api-access-w2cg9\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511148 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-sysconfig\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511175 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-run-openvswitch\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511172 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-env-overrides\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.511278 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.511394 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs podName:343ee524-4530-4307-b119-1c4fb610db6f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:10.011373913 +0000 UTC m=+3.101607101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs") pod "network-metrics-daemon-kgvtw" (UID: "343ee524-4530-4307-b119-1c4fb610db6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511413 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01016934-00c3-4dae-8db6-a118e7c3cbe9-dbus\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.511280 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511452 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-run-ovn\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.511734 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:09.511502 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret podName:01016934-00c3-4dae-8db6-a118e7c3cbe9 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:10.011484902 +0000 UTC m=+3.101718066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret") pod "global-pull-secret-syncer-98h25" (UID: "01016934-00c3-4dae-8db6-a118e7c3cbe9") : object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511497 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-node-log\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511516 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511525 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-run-ovn\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510879 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-socket-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511524 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4vvz\" (UniqueName: \"kubernetes.io/projected/3ebd2686-2acc-4181-9351-9fadd5d6d6a4-kube-api-access-d4vvz\") pod \"iptables-alerter-jf7j4\" (UID: \"3ebd2686-2acc-4181-9351-9fadd5d6d6a4\") " pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511589 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511610 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-sysconfig\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511620 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-sys-fs\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-host\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511674 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-ovn-node-metrics-cert\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17f76060-64b0-4ec1-ae6e-18eaea915761-tmp-dir\") pod \"node-resolver-pw8ln\" (UID: \"17f76060-64b0-4ec1-ae6e-18eaea915761\") " pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511718 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-cni-binary-copy\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511727 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ebd2686-2acc-4181-9351-9fadd5d6d6a4-iptables-alerter-script\") pod \"iptables-alerter-jf7j4\" (UID: \"3ebd2686-2acc-4181-9351-9fadd5d6d6a4\") " pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.512459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511752 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-etc-selinux\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511778 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-os-release\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511780 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-run-openvswitch\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511857 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-host\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.510631 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-run\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.511935 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-os-release\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512038 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-etc-selinux\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512076 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fda8f067-5b21-4d25-9771-77f72ab05e60-serviceca\") pod \"node-ca-6pkk7\" (UID: \"fda8f067-5b21-4d25-9771-77f72ab05e60\") " pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512089 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512091 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17f76060-64b0-4ec1-ae6e-18eaea915761-tmp-dir\") pod \"node-resolver-pw8ln\" (UID: \"17f76060-64b0-4ec1-ae6e-18eaea915761\") " pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512137 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-sys-fs\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512101 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-ovnkube-config\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512165 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512196 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fda8f067-5b21-4d25-9771-77f72ab05e60-host\") pod \"node-ca-6pkk7\" (UID: \"fda8f067-5b21-4d25-9771-77f72ab05e60\") " pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512224 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-cni-bin\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.513159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512265 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-modprobe-d\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512293 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01016934-00c3-4dae-8db6-a118e7c3cbe9-kubelet-config\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512296 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-cni-bin\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512346 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-log-socket\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512365 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01016934-00c3-4dae-8db6-a118e7c3cbe9-kubelet-config\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512369 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ebd2686-2acc-4181-9351-9fadd5d6d6a4-host-slash\") pod \"iptables-alerter-jf7j4\" (UID: \"3ebd2686-2acc-4181-9351-9fadd5d6d6a4\") " pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512400 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ebd2686-2acc-4181-9351-9fadd5d6d6a4-host-slash\") pod \"iptables-alerter-jf7j4\" (UID: \"3ebd2686-2acc-4181-9351-9fadd5d6d6a4\") " pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512407 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-modprobe-d\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512410 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ebd2686-2acc-4181-9351-9fadd5d6d6a4-iptables-alerter-script\") pod \"iptables-alerter-jf7j4\" (UID: \"3ebd2686-2acc-4181-9351-9fadd5d6d6a4\") " pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-device-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512450 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-log-socket\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512453 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fda8f067-5b21-4d25-9771-77f72ab05e60-host\") pod \"node-ca-6pkk7\" (UID: \"fda8f067-5b21-4d25-9771-77f72ab05e60\") " pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512466 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-systemd-units\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512482 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-device-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512497 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-ovnkube-script-lib\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.514274 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512501 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-systemd-units\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512526 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtg8g\" (UniqueName: \"kubernetes.io/projected/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-kube-api-access-vtg8g\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512555 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-kubernetes\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512580 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-sysctl-d\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512624 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-sysctl-conf\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512644 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-kubernetes\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-var-lib-kubelet\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512684 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512714 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npjhj\" (UniqueName: \"kubernetes.io/projected/8512fe14-5aac-4bfb-9e84-e62f0997c20c-kube-api-access-npjhj\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512743 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-etc-openvswitch\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4bm7\" (UniqueName: \"kubernetes.io/projected/343ee524-4530-4307-b119-1c4fb610db6f-kube-api-access-j4bm7\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512824 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-system-cni-dir\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512851 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8512fe14-5aac-4bfb-9e84-e62f0997c20c-tmp\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512962 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-ovnkube-script-lib\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.514983 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.512974 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-sysctl-d\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513014 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513043 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-systemd\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513070 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-run-netns\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513103 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-cni-netd\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513125 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513130 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513188 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-kubelet\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513217 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-registration-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513262 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-cnibin\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513289 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513299 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-run-netns\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513331 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz4f8\" (UniqueName: \"kubernetes.io/projected/17f76060-64b0-4ec1-ae6e-18eaea915761-kube-api-access-nz4f8\") pod \"node-resolver-pw8ln\" (UID: \"17f76060-64b0-4ec1-ae6e-18eaea915761\") " pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513334 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-sysctl-conf\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.515526 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513344 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-cni-netd\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513334 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-kubelet\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513334 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b0c35751-fbd2-4792-987a-fa92ce47b882-registration-dir\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.515963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513379 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-systemd\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.515963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513391 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8512fe14-5aac-4bfb-9e84-e62f0997c20c-var-lib-kubelet\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.515963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513406 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-system-cni-dir\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.515963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513409 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513448 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.515963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513487 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-etc-openvswitch\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.515963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.513497 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-cnibin\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.516539 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.516508 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8512fe14-5aac-4bfb-9e84-e62f0997c20c-etc-tuned\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.516539 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.516529 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8512fe14-5aac-4bfb-9e84-e62f0997c20c-tmp\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.516709 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.516580 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-ovn-node-metrics-cert\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.530325 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.530303 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtg8g\" (UniqueName: \"kubernetes.io/projected/2cb2f081-1c5a-49ed-82bd-2ceb78c3be90-kube-api-access-vtg8g\") pod \"multus-additional-cni-plugins-zpdrq\" (UID: \"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90\") " pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.530767 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.530743 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2cg9\" (UniqueName: \"kubernetes.io/projected/b0c35751-fbd2-4792-987a-fa92ce47b882-kube-api-access-w2cg9\") pod \"aws-ebs-csi-driver-node-7pvjh\" (UID: \"b0c35751-fbd2-4792-987a-fa92ce47b882\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.536631 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.536570 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4vvz\" (UniqueName: \"kubernetes.io/projected/3ebd2686-2acc-4181-9351-9fadd5d6d6a4-kube-api-access-d4vvz\") pod \"iptables-alerter-jf7j4\" (UID: \"3ebd2686-2acc-4181-9351-9fadd5d6d6a4\") " pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.537340 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.537315 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz4f8\" (UniqueName: \"kubernetes.io/projected/17f76060-64b0-4ec1-ae6e-18eaea915761-kube-api-access-nz4f8\") pod \"node-resolver-pw8ln\" (UID: \"17f76060-64b0-4ec1-ae6e-18eaea915761\") " pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.538471 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.538438 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjbs7\" (UniqueName: \"kubernetes.io/projected/fda8f067-5b21-4d25-9771-77f72ab05e60-kube-api-access-rjbs7\") pod \"node-ca-6pkk7\" (UID: \"fda8f067-5b21-4d25-9771-77f72ab05e60\") " pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.538702 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.538679 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npjhj\" (UniqueName: \"kubernetes.io/projected/8512fe14-5aac-4bfb-9e84-e62f0997c20c-kube-api-access-npjhj\") pod \"tuned-9cgzq\" (UID: \"8512fe14-5aac-4bfb-9e84-e62f0997c20c\") " pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.539882 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.539847 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bk9k\" (UniqueName: \"kubernetes.io/projected/cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d-kube-api-access-4bk9k\") pod \"ovnkube-node-v2zh8\" (UID: \"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.540380 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.540362 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4bm7\" (UniqueName: \"kubernetes.io/projected/343ee524-4530-4307-b119-1c4fb610db6f-kube-api-access-j4bm7\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:09.602854 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.602819 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j9sfn" Jul 15 20:59:09.619441 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.619418 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:09.633312 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.633263 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" Jul 15 20:59:09.640885 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.640860 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pw8ln" Jul 15 20:59:09.651641 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.651616 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jf7j4" Jul 15 20:59:09.660641 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.660621 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" Jul 15 20:59:09.668523 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.668499 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" Jul 15 20:59:09.677597 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.677580 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6pkk7" Jul 15 20:59:09.683474 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.683456 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:09.737045 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:09.737009 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 20:59:10.018993 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:10.018953 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:10.019178 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:10.019037 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:10.019178 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:10.019072 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:10.019178 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:10.019155 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 20:59:10.019178 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:10.019193 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:10.019411 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:10.019202 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 20:59:10.019411 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:10.019213 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:10.019411 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:10.019221 2573 projected.go:196] Error preparing data for projected volume kube-api-access-gv5fg for pod openshift-network-diagnostics/network-check-target-z7dvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:10.019411 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:10.019284 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg podName:f37e3221-e7d4-486e-8f9c-5c0d83e702dd nodeName:}" failed. No retries permitted until 2026-07-15 20:59:11.019261717 +0000 UTC m=+4.109494897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gv5fg" (UniqueName: "kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg") pod "network-check-target-z7dvf" (UID: "f37e3221-e7d4-486e-8f9c-5c0d83e702dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:10.019411 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:10.019306 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs podName:343ee524-4530-4307-b119-1c4fb610db6f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:11.019295208 +0000 UTC m=+4.109528372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs") pod "network-metrics-daemon-kgvtw" (UID: "343ee524-4530-4307-b119-1c4fb610db6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:10.019411 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:10.019322 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret podName:01016934-00c3-4dae-8db6-a118e7c3cbe9 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:11.019312314 +0000 UTC m=+4.109545481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret") pod "global-pull-secret-syncer-98h25" (UID: "01016934-00c3-4dae-8db6-a118e7c3cbe9") : object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:10.339458 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:10.339338 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 20:54:09 +0000 UTC" deadline="2028-04-15 16:18:04.75009785 +0000 UTC" Jul 15 20:59:10.339458 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:10.339377 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15355h18m54.410725418s" Jul 15 20:59:10.441492 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:10.441448 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:10.441894 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:10.441588 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:10.577448 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:10.577376 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f76060_64b0_4ec1_ae6e_18eaea915761.slice/crio-36507fe5a951e12022d84985c97f5dbe75af5f26e2d8fa274db7dad2018d604e WatchSource:0}: Error finding container 36507fe5a951e12022d84985c97f5dbe75af5f26e2d8fa274db7dad2018d604e: Status 404 returned error can't find the container with id 36507fe5a951e12022d84985c97f5dbe75af5f26e2d8fa274db7dad2018d604e Jul 15 20:59:10.579579 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:10.579422 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda8f067_5b21_4d25_9771_77f72ab05e60.slice/crio-be2cb00876afd3bb2e147448b9a4655f5cc60e827388a0bf26d01f52b1592a52 WatchSource:0}: Error finding container be2cb00876afd3bb2e147448b9a4655f5cc60e827388a0bf26d01f52b1592a52: Status 404 returned error can't find the container with id be2cb00876afd3bb2e147448b9a4655f5cc60e827388a0bf26d01f52b1592a52 Jul 15 20:59:10.589407 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:10.586072 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9654f9_9400_474b_8978_582043f3d11e.slice/crio-a8201edf4ec5736e4cc1eb34212bc8680bd00d17535c84d38ad6a473a9a83fc8 WatchSource:0}: Error finding container a8201edf4ec5736e4cc1eb34212bc8680bd00d17535c84d38ad6a473a9a83fc8: Status 404 returned error can't find the container with id a8201edf4ec5736e4cc1eb34212bc8680bd00d17535c84d38ad6a473a9a83fc8 Jul 15 20:59:10.606753 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:10.606712 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb2f081_1c5a_49ed_82bd_2ceb78c3be90.slice/crio-989d7416c16a1f585d4763adc9b685a6f51975c3c5e3d1c90e6e641f36e40eef WatchSource:0}: Error finding container 989d7416c16a1f585d4763adc9b685a6f51975c3c5e3d1c90e6e641f36e40eef: Status 404 returned error can't find the container with id 989d7416c16a1f585d4763adc9b685a6f51975c3c5e3d1c90e6e641f36e40eef Jul 15 20:59:10.607652 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:10.607617 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ebd2686_2acc_4181_9351_9fadd5d6d6a4.slice/crio-504ee0be773bcd5d963f0935254efca5d97427b60ee46b65fd0670ad894299e1 WatchSource:0}: Error finding container 504ee0be773bcd5d963f0935254efca5d97427b60ee46b65fd0670ad894299e1: Status 404 returned error can't find the container with id 504ee0be773bcd5d963f0935254efca5d97427b60ee46b65fd0670ad894299e1 Jul 15 20:59:10.609281 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:10.609262 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb55e4a2_16a3_4bcd_8ab1_4bf14d79d72d.slice/crio-62ea99dff48a38d53d21065c60742c76006efda154ddfd24fa7f670c29b1d48d WatchSource:0}: Error finding container 62ea99dff48a38d53d21065c60742c76006efda154ddfd24fa7f670c29b1d48d: Status 404 returned error can't find the container with id 62ea99dff48a38d53d21065c60742c76006efda154ddfd24fa7f670c29b1d48d Jul 15 20:59:10.609929 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:10.609894 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd98705f_b772_418d_ba13_383f7fcda593.slice/crio-6173ad56eacdf78f60a17c5e3d6fa99d50d5f1cd049c4dd1c6688a063fbad4cd WatchSource:0}: Error finding container 6173ad56eacdf78f60a17c5e3d6fa99d50d5f1cd049c4dd1c6688a063fbad4cd: Status 404 returned error can't find the container with id 6173ad56eacdf78f60a17c5e3d6fa99d50d5f1cd049c4dd1c6688a063fbad4cd Jul 15 20:59:10.611577 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:10.611557 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8512fe14_5aac_4bfb_9e84_e62f0997c20c.slice/crio-c67b9884a7bb4c6980c39cac3553c33bc2d30af4b5319521f5f02bd77d0fa04b WatchSource:0}: Error finding container c67b9884a7bb4c6980c39cac3553c33bc2d30af4b5319521f5f02bd77d0fa04b: Status 404 returned error can't find the container with id c67b9884a7bb4c6980c39cac3553c33bc2d30af4b5319521f5f02bd77d0fa04b Jul 15 20:59:11.025731 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.025692 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:11.025890 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.025747 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:11.025890 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.025796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:11.026018 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.025951 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 20:59:11.026018 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.025970 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 20:59:11.026018 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.025994 2573 projected.go:196] Error preparing data for projected volume kube-api-access-gv5fg for pod openshift-network-diagnostics/network-check-target-z7dvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:11.026164 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.026056 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg podName:f37e3221-e7d4-486e-8f9c-5c0d83e702dd nodeName:}" failed. No retries permitted until 2026-07-15 20:59:13.026035942 +0000 UTC m=+6.116269104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gv5fg" (UniqueName: "kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg") pod "network-check-target-z7dvf" (UID: "f37e3221-e7d4-486e-8f9c-5c0d83e702dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:11.026518 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.026494 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:11.026621 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.026556 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs podName:343ee524-4530-4307-b119-1c4fb610db6f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:13.026539061 +0000 UTC m=+6.116772221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs") pod "network-metrics-daemon-kgvtw" (UID: "343ee524-4530-4307-b119-1c4fb610db6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:11.026621 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.026618 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:11.026743 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.026657 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret podName:01016934-00c3-4dae-8db6-a118e7c3cbe9 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:13.026645254 +0000 UTC m=+6.116878413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret") pod "global-pull-secret-syncer-98h25" (UID: "01016934-00c3-4dae-8db6-a118e7c3cbe9") : object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:11.340412 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.340341 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 20:54:09 +0000 UTC" deadline="2028-04-05 20:12:17.389548196 +0000 UTC" Jul 15 20:59:11.340412 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.340408 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15119h13m6.049144384s" Jul 15 20:59:11.442397 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.442367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:11.443003 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.442509 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:11.443003 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.442929 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:11.443135 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:11.443032 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:11.467046 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.466819 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" event={"ID":"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90","Type":"ContainerStarted","Data":"989d7416c16a1f585d4763adc9b685a6f51975c3c5e3d1c90e6e641f36e40eef"} Jul 15 20:59:11.483348 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.483238 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6pkk7" event={"ID":"fda8f067-5b21-4d25-9771-77f72ab05e60","Type":"ContainerStarted","Data":"be2cb00876afd3bb2e147448b9a4655f5cc60e827388a0bf26d01f52b1592a52"} Jul 15 20:59:11.495277 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.495246 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pw8ln" event={"ID":"17f76060-64b0-4ec1-ae6e-18eaea915761","Type":"ContainerStarted","Data":"36507fe5a951e12022d84985c97f5dbe75af5f26e2d8fa274db7dad2018d604e"} Jul 15 20:59:11.511915 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.511884 2573 generic.go:356] "Generic (PLEG): container finished" podID="2d8e486c8c5ef440afb48d0d5efafcdc" containerID="5e16102dc406ff77948b639fe729b7369e194092778d3bf7797678c9e22f360d" exitCode=0 Jul 15 20:59:11.512034 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.511970 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" event={"ID":"2d8e486c8c5ef440afb48d0d5efafcdc","Type":"ContainerDied","Data":"5e16102dc406ff77948b639fe729b7369e194092778d3bf7797678c9e22f360d"} Jul 15 20:59:11.545223 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.545176 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" event={"ID":"8512fe14-5aac-4bfb-9e84-e62f0997c20c","Type":"ContainerStarted","Data":"c67b9884a7bb4c6980c39cac3553c33bc2d30af4b5319521f5f02bd77d0fa04b"} Jul 15 20:59:11.551184 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.551157 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jf7j4" event={"ID":"3ebd2686-2acc-4181-9351-9fadd5d6d6a4","Type":"ContainerStarted","Data":"504ee0be773bcd5d963f0935254efca5d97427b60ee46b65fd0670ad894299e1"} Jul 15 20:59:11.554160 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.554054 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" event={"ID":"b0c35751-fbd2-4792-987a-fa92ce47b882","Type":"ContainerStarted","Data":"4bc03b42eb471039a7dfe0118c74435034b1911329f69255fc6ad59090c95c35"} Jul 15 20:59:11.556538 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.556465 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjhlw" event={"ID":"de9654f9-9400-474b-8978-582043f3d11e","Type":"ContainerStarted","Data":"a8201edf4ec5736e4cc1eb34212bc8680bd00d17535c84d38ad6a473a9a83fc8"} Jul 15 20:59:11.581459 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.581430 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal" event={"ID":"3eb8d182e636c274e1ac9b37ef4dec65","Type":"ContainerStarted","Data":"6e9e162604ded7972a4f7ab117a36b7183af571b90086890b5bda7539c8c7f1e"} Jul 15 20:59:11.591833 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.591762 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j9sfn" event={"ID":"dd98705f-b772-418d-ba13-383f7fcda593","Type":"ContainerStarted","Data":"6173ad56eacdf78f60a17c5e3d6fa99d50d5f1cd049c4dd1c6688a063fbad4cd"} Jul 15 20:59:11.601812 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:11.601709 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerStarted","Data":"62ea99dff48a38d53d21065c60742c76006efda154ddfd24fa7f670c29b1d48d"} Jul 15 20:59:12.441592 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:12.441359 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:12.441592 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:12.441541 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:12.617340 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:12.617302 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" event={"ID":"2d8e486c8c5ef440afb48d0d5efafcdc","Type":"ContainerStarted","Data":"e8c66f8230e8383667950f4bee4237d163d968ed203d9e466848e8805b00c98d"} Jul 15 20:59:12.637295 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:12.636556 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-80.ec2.internal" podStartSLOduration=3.636536489 podStartE2EDuration="3.636536489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:11.600331129 +0000 UTC m=+4.690564313" watchObservedRunningTime="2026-07-15 20:59:12.636536489 +0000 UTC m=+5.726769675" Jul 15 20:59:13.042814 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:13.042772 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:13.042998 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:13.042859 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:13.042998 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:13.042887 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:13.043112 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.043014 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:13.043112 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.043078 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret podName:01016934-00c3-4dae-8db6-a118e7c3cbe9 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:17.043058897 +0000 UTC m=+10.133292058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret") pod "global-pull-secret-syncer-98h25" (UID: "01016934-00c3-4dae-8db6-a118e7c3cbe9") : object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:13.043576 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.043553 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 20:59:13.043658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.043581 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 20:59:13.043658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.043594 2573 projected.go:196] Error preparing data for projected volume kube-api-access-gv5fg for pod openshift-network-diagnostics/network-check-target-z7dvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:13.043658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.043643 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg podName:f37e3221-e7d4-486e-8f9c-5c0d83e702dd nodeName:}" failed. No retries permitted until 2026-07-15 20:59:17.043627595 +0000 UTC m=+10.133860758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gv5fg" (UniqueName: "kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg") pod "network-check-target-z7dvf" (UID: "f37e3221-e7d4-486e-8f9c-5c0d83e702dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:13.043830 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.043706 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:13.043830 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.043739 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs podName:343ee524-4530-4307-b119-1c4fb610db6f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:17.043727481 +0000 UTC m=+10.133960647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs") pod "network-metrics-daemon-kgvtw" (UID: "343ee524-4530-4307-b119-1c4fb610db6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:13.445615 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:13.445543 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:13.445752 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.445663 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:13.445752 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:13.445734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:13.445833 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:13.445799 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:14.441922 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:14.441891 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:14.442323 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:14.442033 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:15.445701 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:15.445583 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:15.446141 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:15.445750 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:15.446247 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:15.445623 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:15.446354 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:15.446307 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:16.442937 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:16.442896 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:16.443108 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:16.443051 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:17.072513 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:17.072455 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:17.072903 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:17.072528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:17.072903 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:17.072583 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:17.072903 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.072781 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 20:59:17.072903 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.072812 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 20:59:17.072903 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.072828 2573 projected.go:196] Error preparing data for projected volume kube-api-access-gv5fg for pod openshift-network-diagnostics/network-check-target-z7dvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:17.072903 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.072897 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg podName:f37e3221-e7d4-486e-8f9c-5c0d83e702dd nodeName:}" failed. No retries permitted until 2026-07-15 20:59:25.072877062 +0000 UTC m=+18.163110226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gv5fg" (UniqueName: "kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg") pod "network-check-target-z7dvf" (UID: "f37e3221-e7d4-486e-8f9c-5c0d83e702dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:17.073197 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.073043 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:17.074073 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.074034 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret podName:01016934-00c3-4dae-8db6-a118e7c3cbe9 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:25.074013183 +0000 UTC m=+18.164246339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret") pod "global-pull-secret-syncer-98h25" (UID: "01016934-00c3-4dae-8db6-a118e7c3cbe9") : object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:17.074192 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.074154 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:17.074244 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.074223 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs podName:343ee524-4530-4307-b119-1c4fb610db6f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:25.074209351 +0000 UTC m=+18.164442509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs") pod "network-metrics-daemon-kgvtw" (UID: "343ee524-4530-4307-b119-1c4fb610db6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:17.442591 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:17.441873 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:17.442591 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.441986 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:17.442591 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:17.442394 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:17.442591 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:17.442487 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:18.441933 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:18.441465 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:18.441933 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:18.441605 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:19.444213 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:19.444184 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:19.444715 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:19.444164 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:19.444715 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:19.444289 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:19.444715 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:19.444367 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:20.441093 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:20.441056 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:20.441274 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:20.441206 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:21.441159 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:21.441123 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:21.441555 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:21.441131 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:21.441555 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:21.441265 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:21.441555 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:21.441312 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:22.441226 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:22.441181 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:22.441654 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:22.441324 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:23.444479 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:23.444449 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:23.444941 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:23.444602 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:23.444941 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:23.444649 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:23.444941 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:23.444728 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:24.441465 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:24.441437 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:24.441691 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:24.441567 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:25.136099 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:25.136059 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:25.136108 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:25.136162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.136232 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.136256 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.136295 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.136302 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret podName:01016934-00c3-4dae-8db6-a118e7c3cbe9 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:41.1362835 +0000 UTC m=+34.226516676 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret") pod "global-pull-secret-syncer-98h25" (UID: "01016934-00c3-4dae-8db6-a118e7c3cbe9") : object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.136311 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.136320 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs podName:343ee524-4530-4307-b119-1c4fb610db6f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:41.136311108 +0000 UTC m=+34.226544265 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs") pod "network-metrics-daemon-kgvtw" (UID: "343ee524-4530-4307-b119-1c4fb610db6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.136323 2573 projected.go:196] Error preparing data for projected volume kube-api-access-gv5fg for pod openshift-network-diagnostics/network-check-target-z7dvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:25.136560 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.136367 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg podName:f37e3221-e7d4-486e-8f9c-5c0d83e702dd nodeName:}" failed. No retries permitted until 2026-07-15 20:59:41.136356056 +0000 UTC m=+34.226589212 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gv5fg" (UniqueName: "kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg") pod "network-check-target-z7dvf" (UID: "f37e3221-e7d4-486e-8f9c-5c0d83e702dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:25.441353 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:25.441288 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:25.441470 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:25.441292 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:25.441470 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.441379 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:25.441470 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:25.441452 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:26.441265 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:26.441227 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:26.441614 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:26.441364 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:27.442448 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:27.442389 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:27.442892 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:27.442509 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:27.442892 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:27.442525 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:27.442892 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:27.442631 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:28.441118 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:28.441085 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:28.441307 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:28.441230 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:29.441321 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.441296 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:29.441705 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.441303 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:29.441705 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:29.441411 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:29.441705 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:29.441484 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:29.657500 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.657255 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" event={"ID":"8512fe14-5aac-4bfb-9e84-e62f0997c20c","Type":"ContainerStarted","Data":"810acb98e1fce9da7314c63aae455860a5480a37e0ebbc0192aeaf2ad5a98e59"} Jul 15 20:59:29.659089 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.659064 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" event={"ID":"b0c35751-fbd2-4792-987a-fa92ce47b882","Type":"ContainerStarted","Data":"15c5244e4b4ed8d6cd5ff3fb8b3417a875ea9118d72426cb6159a1af22f9afc6"} Jul 15 20:59:29.660735 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.660713 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjhlw" event={"ID":"de9654f9-9400-474b-8978-582043f3d11e","Type":"ContainerStarted","Data":"0e72c925593274c5160ea0ddc5d2dead7307957162c67e3af386b88967907cd3"} Jul 15 20:59:29.662365 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.662339 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j9sfn" event={"ID":"dd98705f-b772-418d-ba13-383f7fcda593","Type":"ContainerStarted","Data":"9988bf16c4c77a8920c522fa957b5bcb6e4dca41d790b7ea18a1e4ee2be9418e"} Jul 15 20:59:29.664156 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.664131 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerStarted","Data":"32af38ba3effc34a572e14c976c4fd1b0be9db4e53a9c4970f4d1406124458a1"} Jul 15 20:59:29.665833 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.665807 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" event={"ID":"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90","Type":"ContainerStarted","Data":"9297cd9410914dc6f5175b8c3d4f21bd6a2ca03fabe21ad63c96fe684a3889b8"} Jul 15 20:59:29.667610 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.667587 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6pkk7" event={"ID":"fda8f067-5b21-4d25-9771-77f72ab05e60","Type":"ContainerStarted","Data":"c39ad3029493744d15dc0fcda14df74d653990246ab002f39f3ad473f1d5862f"} Jul 15 20:59:29.668957 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.668908 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pw8ln" event={"ID":"17f76060-64b0-4ec1-ae6e-18eaea915761","Type":"ContainerStarted","Data":"ce622c05614a89481b872d490e12c38d2bc12b5081019ce4bef44d188560599f"} Jul 15 20:59:29.681067 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.681033 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9cgzq" podStartSLOduration=2.853480508 podStartE2EDuration="21.681021927s" totalImagesPullingTime="18.827541419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:29.680877874 +0000 UTC m=+22.771111052" watchObservedRunningTime="2026-07-15 20:59:29.681021927 +0000 UTC m=+22.771255105" Jul 15 20:59:29.681341 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.681320 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-80.ec2.internal" podStartSLOduration=20.681314474 podStartE2EDuration="20.681314474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:12.637634473 +0000 UTC m=+5.727867659" watchObservedRunningTime="2026-07-15 20:59:29.681314474 +0000 UTC m=+22.771547652" Jul 15 20:59:29.698953 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.698918 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-j9sfn" podStartSLOduration=3.819718653 podStartE2EDuration="22.69890655s" totalImagesPullingTime="18.879187897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:29.698716479 +0000 UTC m=+22.788949658" watchObservedRunningTime="2026-07-15 20:59:29.69890655 +0000 UTC m=+22.789139729" Jul 15 20:59:29.743655 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.743598 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pw8ln" podStartSLOduration=2.887158819 podStartE2EDuration="21.743576004s" totalImagesPullingTime="18.856417185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:29.743149482 +0000 UTC m=+22.833382661" watchObservedRunningTime="2026-07-15 20:59:29.743576004 +0000 UTC m=+22.833809183" Jul 15 20:59:29.779572 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.779532 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6pkk7" podStartSLOduration=2.941629304 podStartE2EDuration="21.779515791s" totalImagesPullingTime="18.837886487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:29.763699871 +0000 UTC m=+22.853933048" watchObservedRunningTime="2026-07-15 20:59:29.779515791 +0000 UTC m=+22.869748969" Jul 15 20:59:29.779795 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:29.779769 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zjhlw" podStartSLOduration=3.968331848 podStartE2EDuration="22.779758659s" totalImagesPullingTime="18.811426811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:29.779279995 +0000 UTC m=+22.869513172" watchObservedRunningTime="2026-07-15 20:59:29.779758659 +0000 UTC m=+22.869991837" Jul 15 20:59:30.441164 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.440995 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:30.441393 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:30.441265 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:30.675959 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.675937 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 20:59:30.676273 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.676253 2573 generic.go:356] "Generic (PLEG): container finished" podID="cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d" containerID="179fdba5702a2b21c86f76e3abe2bd7117a3e1d8358061dc45535f71356611e0" exitCode=1 Jul 15 20:59:30.676368 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.676340 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerStarted","Data":"349608527c205ee6c7da2e766c53cee5b621a0848c93976c4a78e191ec30d93a"} Jul 15 20:59:30.676486 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.676466 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerStarted","Data":"99fa3ac4493eb2c32c08524bb302e8db773cf94236fecf8f6489b3cfdef9d19f"} Jul 15 20:59:30.676552 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.676537 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerStarted","Data":"3db8d14914a65655e3ffbd6e96075e64277d91b0ca6b4809d6cde4c91be4ec5c"} Jul 15 20:59:30.676598 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.676586 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerStarted","Data":"6ac04c6b94c1c7b72a1acaa1f0ced5ab081b48fbc3dd1755d24a98d1cc7322d1"} Jul 15 20:59:30.676646 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.676633 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerDied","Data":"179fdba5702a2b21c86f76e3abe2bd7117a3e1d8358061dc45535f71356611e0"} Jul 15 20:59:30.677658 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.677635 2573 generic.go:356] "Generic (PLEG): container finished" podID="2cb2f081-1c5a-49ed-82bd-2ceb78c3be90" containerID="9297cd9410914dc6f5175b8c3d4f21bd6a2ca03fabe21ad63c96fe684a3889b8" exitCode=0 Jul 15 20:59:30.677745 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.677661 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" event={"ID":"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90","Type":"ContainerDied","Data":"9297cd9410914dc6f5175b8c3d4f21bd6a2ca03fabe21ad63c96fe684a3889b8"} Jul 15 20:59:30.679116 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.679003 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jf7j4" event={"ID":"3ebd2686-2acc-4181-9351-9fadd5d6d6a4","Type":"ContainerStarted","Data":"eb78879125feeaf9daccae54f998debef90409aec4bceecdefc316ff851205fe"} Jul 15 20:59:30.775256 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:30.775233 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 20:59:31.064725 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:31.064695 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:31.065286 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:31.065270 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:31.083032 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:31.082992 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jf7j4" podStartSLOduration=4.270342651 podStartE2EDuration="23.082978356s" totalImagesPullingTime="18.812635705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:30.716287874 +0000 UTC m=+23.806521052" watchObservedRunningTime="2026-07-15 20:59:31.082978356 +0000 UTC m=+24.173211533" Jul 15 20:59:31.365164 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:31.365003 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T20:59:30.775251737Z","UUID":"07f0d2bb-e35f-47d5-ac90-25f8b0cc5e07","Handler":null,"Name":"","Endpoint":""} Jul 15 20:59:31.366602 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:31.366581 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 20:59:31.366751 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:31.366615 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 20:59:31.441513 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:31.441485 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:31.442099 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:31.441603 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:31.442099 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:31.441672 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:31.442099 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:31.441755 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:31.683688 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:31.683607 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" event={"ID":"b0c35751-fbd2-4792-987a-fa92ce47b882","Type":"ContainerStarted","Data":"dc2dd3bf1e8aa8bb401eb17f233962b8e3dea46dc287b22365051aa61100a63d"} Jul 15 20:59:32.441445 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:32.441257 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:32.441614 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:32.441536 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:32.687997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:32.687963 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" event={"ID":"b0c35751-fbd2-4792-987a-fa92ce47b882","Type":"ContainerStarted","Data":"ed4f26efe0a175e74bbb32d232185d0669df554b16727459b342418163335753"} Jul 15 20:59:32.696293 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:32.696267 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 20:59:32.696750 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:32.696735 2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 20:59:32.696750 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:32.696728 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerStarted","Data":"8d940b011f1d778c43c6ce8ff231a32e217f16220515272d35f33dd816f94a00"} Jul 15 20:59:32.708801 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:32.708758 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7pvjh" podStartSLOduration=3.755554785 podStartE2EDuration="24.708742966s" totalImagesPullingTime="20.953188181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:32.708538094 +0000 UTC m=+25.798771272" watchObservedRunningTime="2026-07-15 20:59:32.708742966 +0000 UTC m=+25.798976147" Jul 15 20:59:33.441281 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:33.441246 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:33.441281 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:33.441266 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:33.441565 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:33.441389 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:33.441624 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:33.441554 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:34.441207 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:34.441163 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:34.441361 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:34.441328 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:35.441058 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.440864 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:35.441690 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.440912 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:35.441690 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:35.441142 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:35.441690 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:35.441214 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:35.506565 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.506546 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:35.506714 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.506634 2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 20:59:35.507119 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.507093 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zjhlw" Jul 15 20:59:35.706413 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.706390 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 20:59:35.706812 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.706780 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerStarted","Data":"ce48687fc17ff74f7baffcb2933edf223fe68c2c0f86b01c4e0adbdfb24e0021"} Jul 15 20:59:35.707132 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.707046 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:35.707132 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.707128 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:35.707291 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.707196 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:35.707291 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.707257 2573 scope.go:117] "RemoveContainer" containerID="179fdba5702a2b21c86f76e3abe2bd7117a3e1d8358061dc45535f71356611e0" Jul 15 20:59:35.709072 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.709046 2573 generic.go:356] "Generic (PLEG): container finished" podID="2cb2f081-1c5a-49ed-82bd-2ceb78c3be90" containerID="d46d3dfb6ff981c3ab8e9dd34db4703db0e3cef91b0ca20c0c9f0e0346d66c49" exitCode=0 Jul 15 20:59:35.709189 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.709073 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" event={"ID":"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90","Type":"ContainerDied","Data":"d46d3dfb6ff981c3ab8e9dd34db4703db0e3cef91b0ca20c0c9f0e0346d66c49"} Jul 15 20:59:35.723856 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.723836 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:35.724467 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:35.724450 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 20:59:36.441136 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:36.441107 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:36.441599 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:36.441228 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:36.715143 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:36.715084 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 20:59:36.715531 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:36.715507 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" event={"ID":"cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d","Type":"ContainerStarted","Data":"4ef1b4975e5069c38062fc587b96f15176507743c0199b84eea986ff1b3ec32b"} Jul 15 20:59:36.746082 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:36.746041 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" podStartSLOduration=9.881309945 podStartE2EDuration="28.746026936s" totalImagesPullingTime="18.864716991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:36.74553284 +0000 UTC m=+29.835766031" watchObservedRunningTime="2026-07-15 20:59:36.746026936 +0000 UTC m=+29.836260114" Jul 15 20:59:37.108265 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:37.108232 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7dvf"] Jul 15 20:59:37.108497 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:37.108344 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:37.108497 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:37.108435 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:37.110579 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:37.110555 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-98h25"] Jul 15 20:59:37.110683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:37.110645 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:37.110733 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:37.110718 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:37.128410 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:37.128391 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kgvtw"] Jul 15 20:59:37.128528 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:37.128464 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:37.128589 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:37.128570 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:37.719777 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:37.719742 2573 generic.go:356] "Generic (PLEG): container finished" podID="2cb2f081-1c5a-49ed-82bd-2ceb78c3be90" containerID="42014356c577e978aa25b678ab2736d8cd27284df54d5e40af325532f49234bc" exitCode=0 Jul 15 20:59:37.720238 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:37.719826 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" event={"ID":"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90","Type":"ContainerDied","Data":"42014356c577e978aa25b678ab2736d8cd27284df54d5e40af325532f49234bc"} Jul 15 20:59:38.441687 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:38.441502 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:38.441867 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:38.441524 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:38.441867 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:38.441754 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:38.441986 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:38.441860 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:38.441986 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:38.441585 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:38.441986 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:38.441968 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:39.726370 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:39.726335 2573 generic.go:356] "Generic (PLEG): container finished" podID="2cb2f081-1c5a-49ed-82bd-2ceb78c3be90" containerID="521e9d6ebc74fa7a87e73e278b4c2882cecb52790f13c1fe833317614f1ab410" exitCode=0 Jul 15 20:59:39.726976 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:39.726385 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" event={"ID":"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90","Type":"ContainerDied","Data":"521e9d6ebc74fa7a87e73e278b4c2882cecb52790f13c1fe833317614f1ab410"} Jul 15 20:59:40.440758 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:40.440728 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:40.440758 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:40.440751 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:40.441007 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:40.440728 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:40.441007 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:40.440851 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7dvf" podUID="f37e3221-e7d4-486e-8f9c-5c0d83e702dd" Jul 15 20:59:40.441007 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:40.440956 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-98h25" podUID="01016934-00c3-4dae-8db6-a118e7c3cbe9" Jul 15 20:59:40.441143 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:40.441075 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kgvtw" podUID="343ee524-4530-4307-b119-1c4fb610db6f" Jul 15 20:59:41.157024 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:41.156987 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:41.157024 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:41.157023 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:41.157658 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:41.157062 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:41.157658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:41.157187 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 20:59:41.157658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:41.157190 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:41.157658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:41.157205 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 20:59:41.157658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:41.157225 2573 projected.go:196] Error preparing data for projected volume kube-api-access-gv5fg for pod openshift-network-diagnostics/network-check-target-z7dvf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:41.157658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:41.157188 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:41.157658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:41.157266 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret podName:01016934-00c3-4dae-8db6-a118e7c3cbe9 nodeName:}" failed. No retries permitted until 2026-07-15 21:00:13.157244954 +0000 UTC m=+66.247478114 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret") pod "global-pull-secret-syncer-98h25" (UID: "01016934-00c3-4dae-8db6-a118e7c3cbe9") : object "kube-system"/"original-pull-secret" not registered Jul 15 20:59:41.157658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:41.157285 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg podName:f37e3221-e7d4-486e-8f9c-5c0d83e702dd nodeName:}" failed. No retries permitted until 2026-07-15 21:00:13.157275634 +0000 UTC m=+66.247508796 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gv5fg" (UniqueName: "kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg") pod "network-check-target-z7dvf" (UID: "f37e3221-e7d4-486e-8f9c-5c0d83e702dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 20:59:41.157658 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:41.157302 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs podName:343ee524-4530-4307-b119-1c4fb610db6f nodeName:}" failed. No retries permitted until 2026-07-15 21:00:13.157293125 +0000 UTC m=+66.247526284 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs") pod "network-metrics-daemon-kgvtw" (UID: "343ee524-4530-4307-b119-1c4fb610db6f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 20:59:42.193157 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.193087 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-80.ec2.internal" event="NodeReady" Jul 15 20:59:42.193559 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.193250 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 20:59:42.230017 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.229989 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77f7469478-q45vb"] Jul 15 20:59:42.232044 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.232023 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.235372 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.235350 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 20:59:42.235471 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.235355 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 20:59:42.235471 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.235436 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 20:59:42.235868 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.235848 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pndn8\"" Jul 15 20:59:42.244569 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.244544 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 20:59:42.247447 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.247423 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77f7469478-q45vb"] Jul 15 20:59:42.251444 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.251200 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jmfmx"] Jul 15 20:59:42.253295 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.253277 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5jlnp"] Jul 15 20:59:42.253430 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.253411 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.255331 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.255315 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5jlnp" Jul 15 20:59:42.256801 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.256783 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 20:59:42.257144 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.257126 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9rgfh\"" Jul 15 20:59:42.257420 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.257405 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 20:59:42.257824 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.257808 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cjhbj\"" Jul 15 20:59:42.257937 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.257920 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 20:59:42.258093 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.258076 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 20:59:42.258446 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.258430 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 20:59:42.264795 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.264687 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5jlnp"] Jul 15 20:59:42.265606 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.265550 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jmfmx"] Jul 15 20:59:42.365401 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365372 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-ca-trust-extracted\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.365559 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365431 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-image-registry-private-configuration\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.365559 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365456 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjz4d\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-kube-api-access-tjz4d\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.365559 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365479 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.365664 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365628 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1-cert\") pod \"ingress-canary-5jlnp\" (UID: \"4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1\") " pod="openshift-ingress-canary/ingress-canary-5jlnp" Jul 15 20:59:42.365704 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365665 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-certificates\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.365704 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365687 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-trusted-ca\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.365793 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365724 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjnkt\" (UniqueName: \"kubernetes.io/projected/4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1-kube-api-access-vjnkt\") pod \"ingress-canary-5jlnp\" (UID: \"4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1\") " pod="openshift-ingress-canary/ingress-canary-5jlnp" Jul 15 20:59:42.365793 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365759 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-bound-sa-token\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.365793 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365788 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqx4j\" (UniqueName: \"kubernetes.io/projected/47f9f4d6-0555-4123-b64b-5f753c9e1160-kube-api-access-wqx4j\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.365926 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365815 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.365926 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365838 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-installation-pull-secrets\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.366032 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365923 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47f9f4d6-0555-4123-b64b-5f753c9e1160-config-volume\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.366032 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.365951 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47f9f4d6-0555-4123-b64b-5f753c9e1160-tmp-dir\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.441014 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.440981 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 20:59:42.441189 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.441019 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 20:59:42.441189 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.441024 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 20:59:42.444387 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.444110 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 20:59:42.444387 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.444117 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 20:59:42.444387 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.444116 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 20:59:42.444387 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.444128 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 20:59:42.444387 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.444201 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ffp72\"" Jul 15 20:59:42.444387 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.444118 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jnsrt\"" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.466838 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-ca-trust-extracted\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.466900 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-image-registry-private-configuration\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.466926 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjz4d\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-kube-api-access-tjz4d\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.466954 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.466978 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1-cert\") pod \"ingress-canary-5jlnp\" (UID: \"4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1\") " pod="openshift-ingress-canary/ingress-canary-5jlnp" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467006 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-certificates\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467031 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-trusted-ca\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjnkt\" (UniqueName: \"kubernetes.io/projected/4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1-kube-api-access-vjnkt\") pod \"ingress-canary-5jlnp\" (UID: \"4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1\") " pod="openshift-ingress-canary/ingress-canary-5jlnp" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467086 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-bound-sa-token\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467114 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqx4j\" (UniqueName: \"kubernetes.io/projected/47f9f4d6-0555-4123-b64b-5f753c9e1160-kube-api-access-wqx4j\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467140 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467165 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-installation-pull-secrets\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467233 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47f9f4d6-0555-4123-b64b-5f753c9e1160-config-volume\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.467600 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467257 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47f9f4d6-0555-4123-b64b-5f753c9e1160-tmp-dir\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.468335 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.467432 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 20:59:42.468335 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.467450 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f7469478-q45vb: secret "image-registry-tls" not found Jul 15 20:59:42.468335 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.467505 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls podName:f8a2ffcc-698c-43ae-8659-ed37b95f5fd5 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:42.967485887 +0000 UTC m=+36.057719063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls") pod "image-registry-77f7469478-q45vb" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5") : secret "image-registry-tls" not found Jul 15 20:59:42.468335 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.467672 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 20:59:42.468335 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.467882 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls podName:47f9f4d6-0555-4123-b64b-5f753c9e1160 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:42.967867316 +0000 UTC m=+36.058100478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls") pod "dns-default-jmfmx" (UID: "47f9f4d6-0555-4123-b64b-5f753c9e1160") : secret "dns-default-metrics-tls" not found Jul 15 20:59:42.468335 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.467937 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-certificates\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.468335 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.468308 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47f9f4d6-0555-4123-b64b-5f753c9e1160-tmp-dir\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.468683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.468348 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-ca-trust-extracted\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.468683 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.468504 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47f9f4d6-0555-4123-b64b-5f753c9e1160-config-volume\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.469518 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.469495 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-trusted-ca\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.472004 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.471982 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-installation-pull-secrets\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.472108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.472016 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1-cert\") pod \"ingress-canary-5jlnp\" (UID: \"4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1\") " pod="openshift-ingress-canary/ingress-canary-5jlnp" Jul 15 20:59:42.472108 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.471987 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-image-registry-private-configuration\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.480043 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.480015 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjz4d\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-kube-api-access-tjz4d\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.481153 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.481111 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqx4j\" (UniqueName: \"kubernetes.io/projected/47f9f4d6-0555-4123-b64b-5f753c9e1160-kube-api-access-wqx4j\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.481513 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.481495 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-bound-sa-token\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.481684 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.481669 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjnkt\" (UniqueName: \"kubernetes.io/projected/4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1-kube-api-access-vjnkt\") pod \"ingress-canary-5jlnp\" (UID: \"4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1\") " pod="openshift-ingress-canary/ingress-canary-5jlnp" Jul 15 20:59:42.571538 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.571503 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5jlnp" Jul 15 20:59:42.878718 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.878686 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-xsxss"] Jul 15 20:59:42.880816 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.880795 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:42.883623 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.883587 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 15 20:59:42.883713 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.883630 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 20:59:42.883713 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.883641 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 20:59:42.883851 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.883717 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vvr6c\"" Jul 15 20:59:42.883851 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.883769 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 15 20:59:42.889211 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.889195 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 15 20:59:42.892824 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.892805 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-xsxss"] Jul 15 20:59:42.972362 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.972325 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-tmp\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:42.972513 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.972367 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pptbp\" (UniqueName: \"kubernetes.io/projected/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-kube-api-access-pptbp\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:42.972513 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.972401 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:42.972513 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.972424 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:42.972513 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.972465 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-serving-cert\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:42.972513 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.972499 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:42.972886 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.972535 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-snapshots\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:42.973009 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.972931 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:42.973106 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.973085 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 20:59:42.973161 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.973138 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 20:59:42.973161 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.973153 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f7469478-q45vb: secret "image-registry-tls" not found Jul 15 20:59:42.973279 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.973216 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls podName:47f9f4d6-0555-4123-b64b-5f753c9e1160 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:43.973187256 +0000 UTC m=+37.063420428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls") pod "dns-default-jmfmx" (UID: "47f9f4d6-0555-4123-b64b-5f753c9e1160") : secret "dns-default-metrics-tls" not found Jul 15 20:59:42.973279 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:42.973262 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls podName:f8a2ffcc-698c-43ae-8659-ed37b95f5fd5 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:43.973250575 +0000 UTC m=+37.063483745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls") pod "image-registry-77f7469478-q45vb" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5") : secret "image-registry-tls" not found Jul 15 20:59:42.982893 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.982870 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp"] Jul 15 20:59:42.985398 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.985362 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l"] Jul 15 20:59:42.985911 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.985624 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp" Jul 15 20:59:42.987154 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.987134 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh"] Jul 15 20:59:42.987268 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.987219 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:42.988673 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.988654 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 20:59:42.988916 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.988892 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-jwz4p"] Jul 15 20:59:42.989018 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.988998 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:42.989326 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.989138 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ktr2k\"" Jul 15 20:59:42.989611 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.989594 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:59:42.989942 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.989924 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 20:59:42.990935 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.990885 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 20:59:42.990935 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.990885 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k9ll8\"" Jul 15 20:59:42.991229 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.991210 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7"] Jul 15 20:59:42.991346 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.991273 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:59:42.991346 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.991320 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:42.991828 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.991799 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:59:42.991915 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.991852 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wzs8m\"" Jul 15 20:59:42.991915 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.991857 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 20:59:42.991915 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.991863 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 20:59:42.991915 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.991863 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 20:59:42.993666 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.993649 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 20:59:42.995987 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.995967 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:42.997725 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.997708 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp"] Jul 15 20:59:42.998014 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.997995 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 20:59:42.998120 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.998106 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:59:42.998509 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.998493 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 20:59:42.998843 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.998826 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 20:59:42.998963 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:42.998945 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5swh9\"" Jul 15 20:59:43.001375 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.001332 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 20:59:43.001595 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.001576 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 20:59:43.001789 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.001773 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 20:59:43.002023 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.002006 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rxn57\"" Jul 15 20:59:43.005638 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.005615 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh"] Jul 15 20:59:43.006611 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.006586 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7"] Jul 15 20:59:43.007497 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.007478 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-jwz4p"] Jul 15 20:59:43.008392 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.008373 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 20:59:43.019256 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.019238 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l"] Jul 15 20:59:43.073603 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.073769 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:43.073769 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073649 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/84e688bb-4303-4ac4-b180-3ae21fa17823-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:43.073769 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073717 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af1545f9-cf6b-4ba4-a00e-2c74949572c3-serving-cert\") pod \"service-ca-operator-7695fc6c8c-p2xkh\" (UID: \"af1545f9-cf6b-4ba4-a00e-2c74949572c3\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.073922 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073772 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pptbp\" (UniqueName: \"kubernetes.io/projected/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-kube-api-access-pptbp\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.073922 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073822 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cb05969b-c83d-4c37-8c63-1221309d2f8b-config\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.073922 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073866 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-tmp\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.073922 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073892 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clxxm\" (UniqueName: \"kubernetes.io/projected/88ea6a7b-188f-41a2-818c-f5780e71b44f-kube-api-access-clxxm\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:43.073922 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073924 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-serving-cert\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.074149 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073940 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npvgb\" (UniqueName: \"kubernetes.io/projected/79083cd2-3480-42d0-bdf5-a5719b9f8995-kube-api-access-npvgb\") pod \"volume-data-source-validator-78495597f6-rs6fp\" (UID: \"79083cd2-3480-42d0-bdf5-a5719b9f8995\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp" Jul 15 20:59:43.074149 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.073967 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-snapshots\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.074149 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074018 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rshqd\" (UniqueName: \"kubernetes.io/projected/af1545f9-cf6b-4ba4-a00e-2c74949572c3-kube-api-access-rshqd\") pod \"service-ca-operator-7695fc6c8c-p2xkh\" (UID: \"af1545f9-cf6b-4ba4-a00e-2c74949572c3\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.074300 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074210 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-tmp\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.074520 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074493 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.074598 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074538 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-snapshots\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.074598 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.074694 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074613 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:43.074694 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074644 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb05969b-c83d-4c37-8c63-1221309d2f8b-serving-cert\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.074694 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074668 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af1545f9-cf6b-4ba4-a00e-2c74949572c3-config\") pod \"service-ca-operator-7695fc6c8c-p2xkh\" (UID: \"af1545f9-cf6b-4ba4-a00e-2c74949572c3\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.074844 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074772 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb05969b-c83d-4c37-8c63-1221309d2f8b-trusted-ca\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.074844 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074820 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbbxg\" (UniqueName: \"kubernetes.io/projected/84e688bb-4303-4ac4-b180-3ae21fa17823-kube-api-access-kbbxg\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:43.074930 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.074848 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmvlp\" (UniqueName: \"kubernetes.io/projected/cb05969b-c83d-4c37-8c63-1221309d2f8b-kube-api-access-vmvlp\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.075066 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.075041 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.077093 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.077075 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-serving-cert\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.086197 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.083724 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pptbp\" (UniqueName: \"kubernetes.io/projected/3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88-kube-api-access-pptbp\") pod \"insights-operator-68d9dd94cf-xsxss\" (UID: \"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88\") " pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.090033 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.090009 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-585bb7b99d-xcp26"] Jul 15 20:59:43.093205 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.093185 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.096134 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.096114 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 20:59:43.096240 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.096160 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 20:59:43.096283 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.096247 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-l249s\"" Jul 15 20:59:43.096334 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.096201 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 20:59:43.096334 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.096121 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 20:59:43.096472 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.096451 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 20:59:43.096586 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.096552 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 20:59:43.103143 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.103123 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-585bb7b99d-xcp26"] Jul 15 20:59:43.175441 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:43.175441 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175406 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb05969b-c83d-4c37-8c63-1221309d2f8b-serving-cert\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.175441 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175435 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af1545f9-cf6b-4ba4-a00e-2c74949572c3-config\") pod \"service-ca-operator-7695fc6c8c-p2xkh\" (UID: \"af1545f9-cf6b-4ba4-a00e-2c74949572c3\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.175685 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175471 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb05969b-c83d-4c37-8c63-1221309d2f8b-trusted-ca\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.175685 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175499 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-stats-auth\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.175685 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.175532 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 20:59:43.175685 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175554 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbbxg\" (UniqueName: \"kubernetes.io/projected/84e688bb-4303-4ac4-b180-3ae21fa17823-kube-api-access-kbbxg\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:43.175685 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175580 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmvlp\" (UniqueName: \"kubernetes.io/projected/cb05969b-c83d-4c37-8c63-1221309d2f8b-kube-api-access-vmvlp\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.175685 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.175612 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls podName:88ea6a7b-188f-41a2-818c-f5780e71b44f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:43.675592147 +0000 UTC m=+36.765825317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls") pod "cluster-samples-operator-65db858894-j2w2l" (UID: "88ea6a7b-188f-41a2-818c-f5780e71b44f") : secret "samples-operator-tls" not found Jul 15 20:59:43.175685 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175655 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-default-certificate\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175696 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175724 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/84e688bb-4303-4ac4-b180-3ae21fa17823-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175751 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175805 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af1545f9-cf6b-4ba4-a00e-2c74949572c3-serving-cert\") pod \"service-ca-operator-7695fc6c8c-p2xkh\" (UID: \"af1545f9-cf6b-4ba4-a00e-2c74949572c3\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175834 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175871 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv4cb\" (UniqueName: \"kubernetes.io/projected/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-kube-api-access-fv4cb\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175901 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cb05969b-c83d-4c37-8c63-1221309d2f8b-config\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.175933 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175952 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clxxm\" (UniqueName: \"kubernetes.io/projected/88ea6a7b-188f-41a2-818c-f5780e71b44f-kube-api-access-clxxm\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:43.175997 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.175993 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npvgb\" (UniqueName: \"kubernetes.io/projected/79083cd2-3480-42d0-bdf5-a5719b9f8995-kube-api-access-npvgb\") pod \"volume-data-source-validator-78495597f6-rs6fp\" (UID: \"79083cd2-3480-42d0-bdf5-a5719b9f8995\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp" Jul 15 20:59:43.176475 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.176008 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af1545f9-cf6b-4ba4-a00e-2c74949572c3-config\") pod \"service-ca-operator-7695fc6c8c-p2xkh\" (UID: \"af1545f9-cf6b-4ba4-a00e-2c74949572c3\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.176475 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.176028 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rshqd\" (UniqueName: \"kubernetes.io/projected/af1545f9-cf6b-4ba4-a00e-2c74949572c3-kube-api-access-rshqd\") pod \"service-ca-operator-7695fc6c8c-p2xkh\" (UID: \"af1545f9-cf6b-4ba4-a00e-2c74949572c3\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.176475 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.176095 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls podName:84e688bb-4303-4ac4-b180-3ae21fa17823 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:43.67607494 +0000 UTC m=+36.766308100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6pcv7" (UID: "84e688bb-4303-4ac4-b180-3ae21fa17823") : secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:43.176859 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.176801 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/84e688bb-4303-4ac4-b180-3ae21fa17823-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:43.176957 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.176887 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cb05969b-c83d-4c37-8c63-1221309d2f8b-config\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.177029 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.176979 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cb05969b-c83d-4c37-8c63-1221309d2f8b-trusted-ca\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.178253 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.178234 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb05969b-c83d-4c37-8c63-1221309d2f8b-serving-cert\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.178674 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.178653 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af1545f9-cf6b-4ba4-a00e-2c74949572c3-serving-cert\") pod \"service-ca-operator-7695fc6c8c-p2xkh\" (UID: \"af1545f9-cf6b-4ba4-a00e-2c74949572c3\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.179925 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.179904 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc"] Jul 15 20:59:43.182709 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.182691 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.186097 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.186077 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 15 20:59:43.186980 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.186337 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 15 20:59:43.186980 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.186581 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:59:43.186980 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.186857 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 15 20:59:43.187207 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.187062 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jftr6\"" Jul 15 20:59:43.188461 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.188442 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rshqd\" (UniqueName: \"kubernetes.io/projected/af1545f9-cf6b-4ba4-a00e-2c74949572c3-kube-api-access-rshqd\") pod \"service-ca-operator-7695fc6c8c-p2xkh\" (UID: \"af1545f9-cf6b-4ba4-a00e-2c74949572c3\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.188943 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.188922 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npvgb\" (UniqueName: \"kubernetes.io/projected/79083cd2-3480-42d0-bdf5-a5719b9f8995-kube-api-access-npvgb\") pod \"volume-data-source-validator-78495597f6-rs6fp\" (UID: \"79083cd2-3480-42d0-bdf5-a5719b9f8995\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp" Jul 15 20:59:43.189114 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.188985 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmvlp\" (UniqueName: \"kubernetes.io/projected/cb05969b-c83d-4c37-8c63-1221309d2f8b-kube-api-access-vmvlp\") pod \"console-operator-7866497494-jwz4p\" (UID: \"cb05969b-c83d-4c37-8c63-1221309d2f8b\") " pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.189556 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.189536 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbbxg\" (UniqueName: \"kubernetes.io/projected/84e688bb-4303-4ac4-b180-3ae21fa17823-kube-api-access-kbbxg\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:43.190799 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.190635 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clxxm\" (UniqueName: \"kubernetes.io/projected/88ea6a7b-188f-41a2-818c-f5780e71b44f-kube-api-access-clxxm\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:43.192510 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.192492 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" Jul 15 20:59:43.195432 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.195409 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc"] Jul 15 20:59:43.277230 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.277185 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-stats-auth\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.277230 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.277233 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a35d610b-d08f-47d4-99c9-9f6aa55fa6ff-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rvmvc\" (UID: \"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.277467 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.277284 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-default-certificate\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.277467 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.277313 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a35d610b-d08f-47d4-99c9-9f6aa55fa6ff-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rvmvc\" (UID: \"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.277467 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.277355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.277467 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.277408 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.277467 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.277434 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv4cb\" (UniqueName: \"kubernetes.io/projected/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-kube-api-access-fv4cb\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.277684 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.277467 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 20:59:43.277684 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.277502 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgqg6\" (UniqueName: \"kubernetes.io/projected/a35d610b-d08f-47d4-99c9-9f6aa55fa6ff-kube-api-access-mgqg6\") pod \"kube-storage-version-migrator-operator-799cb7f958-rvmvc\" (UID: \"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.277684 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.277536 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:43.777515616 +0000 UTC m=+36.867748793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : secret "router-metrics-certs-default" not found Jul 15 20:59:43.277684 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.277635 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:43.7776173 +0000 UTC m=+36.867850465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : configmap references non-existent config key: service-ca.crt Jul 15 20:59:43.280145 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.280125 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-default-certificate\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.280266 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.280245 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-stats-auth\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.286667 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.286643 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv4cb\" (UniqueName: \"kubernetes.io/projected/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-kube-api-access-fv4cb\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.297204 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.297184 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp" Jul 15 20:59:43.314540 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.314518 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" Jul 15 20:59:43.322262 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.322241 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:43.378926 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.378900 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgqg6\" (UniqueName: \"kubernetes.io/projected/a35d610b-d08f-47d4-99c9-9f6aa55fa6ff-kube-api-access-mgqg6\") pod \"kube-storage-version-migrator-operator-799cb7f958-rvmvc\" (UID: \"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.379027 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.378964 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a35d610b-d08f-47d4-99c9-9f6aa55fa6ff-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rvmvc\" (UID: \"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.379027 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.379005 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a35d610b-d08f-47d4-99c9-9f6aa55fa6ff-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rvmvc\" (UID: \"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.379580 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.379556 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a35d610b-d08f-47d4-99c9-9f6aa55fa6ff-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-rvmvc\" (UID: \"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.381327 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.381303 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a35d610b-d08f-47d4-99c9-9f6aa55fa6ff-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-rvmvc\" (UID: \"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.388666 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.388648 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgqg6\" (UniqueName: \"kubernetes.io/projected/a35d610b-d08f-47d4-99c9-9f6aa55fa6ff-kube-api-access-mgqg6\") pod \"kube-storage-version-migrator-operator-799cb7f958-rvmvc\" (UID: \"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.493371 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.493300 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" Jul 15 20:59:43.682953 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.682914 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:43.683111 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.683001 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:43.683111 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.683081 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 20:59:43.683258 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.683139 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:43.683258 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.683159 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls podName:88ea6a7b-188f-41a2-818c-f5780e71b44f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:44.68313896 +0000 UTC m=+37.773372136 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls") pod "cluster-samples-operator-65db858894-j2w2l" (UID: "88ea6a7b-188f-41a2-818c-f5780e71b44f") : secret "samples-operator-tls" not found Jul 15 20:59:43.683258 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.683211 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls podName:84e688bb-4303-4ac4-b180-3ae21fa17823 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:44.683194437 +0000 UTC m=+37.773427596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6pcv7" (UID: "84e688bb-4303-4ac4-b180-3ae21fa17823") : secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:43.783805 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.783771 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.784006 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.783856 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:43.784006 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.783897 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 20:59:43.784006 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.783973 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:44.78395402 +0000 UTC m=+37.874187181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : secret "router-metrics-certs-default" not found Jul 15 20:59:43.784006 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.784000 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:44.783982669 +0000 UTC m=+37.874215859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : configmap references non-existent config key: service-ca.crt Jul 15 20:59:43.985830 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.985796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:43.985965 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:43.985862 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:43.986067 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.986016 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 20:59:43.986067 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.986031 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f7469478-q45vb: secret "image-registry-tls" not found Jul 15 20:59:43.986231 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.986213 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 20:59:43.986315 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.986296 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls podName:47f9f4d6-0555-4123-b64b-5f753c9e1160 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:45.986274958 +0000 UTC m=+39.076508115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls") pod "dns-default-jmfmx" (UID: "47f9f4d6-0555-4123-b64b-5f753c9e1160") : secret "dns-default-metrics-tls" not found Jul 15 20:59:43.986758 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:43.986716 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls podName:f8a2ffcc-698c-43ae-8659-ed37b95f5fd5 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:45.986684238 +0000 UTC m=+39.076917423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls") pod "image-registry-77f7469478-q45vb" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5") : secret "image-registry-tls" not found Jul 15 20:59:44.162076 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.162003 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp"] Jul 15 20:59:44.167602 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:44.167569 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79083cd2_3480_42d0_bdf5_a5719b9f8995.slice/crio-32c6a056f112425074cc94eea80d5c7b50644e204e458100024a37fab26fcc90 WatchSource:0}: Error finding container 32c6a056f112425074cc94eea80d5c7b50644e204e458100024a37fab26fcc90: Status 404 returned error can't find the container with id 32c6a056f112425074cc94eea80d5c7b50644e204e458100024a37fab26fcc90 Jul 15 20:59:44.175632 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.175556 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc"] Jul 15 20:59:44.187236 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.187214 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5jlnp"] Jul 15 20:59:44.190521 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.190500 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-jwz4p"] Jul 15 20:59:44.192974 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.192951 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-xsxss"] Jul 15 20:59:44.195840 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.195818 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh"] Jul 15 20:59:44.201338 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:44.201309 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec81d32_4e52_4f6e_a9a9_a4c0d19c0ef1.slice/crio-98f7a11bb482b2d67efdcf3f1e5f00ee646545863724c2a8de24447fe97787c6 WatchSource:0}: Error finding container 98f7a11bb482b2d67efdcf3f1e5f00ee646545863724c2a8de24447fe97787c6: Status 404 returned error can't find the container with id 98f7a11bb482b2d67efdcf3f1e5f00ee646545863724c2a8de24447fe97787c6 Jul 15 20:59:44.202087 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:44.202057 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35d610b_d08f_47d4_99c9_9f6aa55fa6ff.slice/crio-5c805d6ae01b31291621c2048f061cd8ef3a864b9b144cc6765bf9901d8abe4a WatchSource:0}: Error finding container 5c805d6ae01b31291621c2048f061cd8ef3a864b9b144cc6765bf9901d8abe4a: Status 404 returned error can't find the container with id 5c805d6ae01b31291621c2048f061cd8ef3a864b9b144cc6765bf9901d8abe4a Jul 15 20:59:44.203345 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:44.203121 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb05969b_c83d_4c37_8c63_1221309d2f8b.slice/crio-2112bceada11ceb56ee65c5ca25591c4b402f5afcf4e6c524ef17f4920eca721 WatchSource:0}: Error finding container 2112bceada11ceb56ee65c5ca25591c4b402f5afcf4e6c524ef17f4920eca721: Status 404 returned error can't find the container with id 2112bceada11ceb56ee65c5ca25591c4b402f5afcf4e6c524ef17f4920eca721 Jul 15 20:59:44.204184 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:44.204151 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c460d9e_76ff_4e7f_8d04_f90f2b8c2b88.slice/crio-57d33e20b779d02d23ffa437e8a74be9d171b4f563d945d28faeba9877dec3d0 WatchSource:0}: Error finding container 57d33e20b779d02d23ffa437e8a74be9d171b4f563d945d28faeba9877dec3d0: Status 404 returned error can't find the container with id 57d33e20b779d02d23ffa437e8a74be9d171b4f563d945d28faeba9877dec3d0 Jul 15 20:59:44.204675 ip-10-0-135-80 kubenswrapper[2573]: W0715 20:59:44.204503 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1545f9_cf6b_4ba4_a00e_2c74949572c3.slice/crio-19aaaa2f0148d29bfb70f952f37657596916bb5395acfd7b7f368ba5f805e883 WatchSource:0}: Error finding container 19aaaa2f0148d29bfb70f952f37657596916bb5395acfd7b7f368ba5f805e883: Status 404 returned error can't find the container with id 19aaaa2f0148d29bfb70f952f37657596916bb5395acfd7b7f368ba5f805e883 Jul 15 20:59:44.696153 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.696117 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:44.696343 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.696200 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:44.696343 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:44.696289 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 20:59:44.696465 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:44.696356 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls podName:88ea6a7b-188f-41a2-818c-f5780e71b44f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:46.696341688 +0000 UTC m=+39.786574849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls") pod "cluster-samples-operator-65db858894-j2w2l" (UID: "88ea6a7b-188f-41a2-818c-f5780e71b44f") : secret "samples-operator-tls" not found Jul 15 20:59:44.696465 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:44.696371 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:44.696465 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:44.696415 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls podName:84e688bb-4303-4ac4-b180-3ae21fa17823 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:46.696404113 +0000 UTC m=+39.786637273 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6pcv7" (UID: "84e688bb-4303-4ac4-b180-3ae21fa17823") : secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:44.740946 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.740829 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" event={"ID":"af1545f9-cf6b-4ba4-a00e-2c74949572c3","Type":"ContainerStarted","Data":"19aaaa2f0148d29bfb70f952f37657596916bb5395acfd7b7f368ba5f805e883"} Jul 15 20:59:44.742604 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.742537 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jwz4p" event={"ID":"cb05969b-c83d-4c37-8c63-1221309d2f8b","Type":"ContainerStarted","Data":"2112bceada11ceb56ee65c5ca25591c4b402f5afcf4e6c524ef17f4920eca721"} Jul 15 20:59:44.745247 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.745196 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5jlnp" event={"ID":"4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1","Type":"ContainerStarted","Data":"98f7a11bb482b2d67efdcf3f1e5f00ee646545863724c2a8de24447fe97787c6"} Jul 15 20:59:44.746864 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.746810 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" event={"ID":"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff","Type":"ContainerStarted","Data":"5c805d6ae01b31291621c2048f061cd8ef3a864b9b144cc6765bf9901d8abe4a"} Jul 15 20:59:44.750251 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.750202 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp" event={"ID":"79083cd2-3480-42d0-bdf5-a5719b9f8995","Type":"ContainerStarted","Data":"32c6a056f112425074cc94eea80d5c7b50644e204e458100024a37fab26fcc90"} Jul 15 20:59:44.752756 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.752735 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" event={"ID":"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88","Type":"ContainerStarted","Data":"57d33e20b779d02d23ffa437e8a74be9d171b4f563d945d28faeba9877dec3d0"} Jul 15 20:59:44.758020 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.757992 2573 generic.go:356] "Generic (PLEG): container finished" podID="2cb2f081-1c5a-49ed-82bd-2ceb78c3be90" containerID="eaac20a35c161f74decc5e48d8bddcaa25e4d0173d29c808cda702771d8c8c5d" exitCode=0 Jul 15 20:59:44.758111 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.758032 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" event={"ID":"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90","Type":"ContainerDied","Data":"eaac20a35c161f74decc5e48d8bddcaa25e4d0173d29c808cda702771d8c8c5d"} Jul 15 20:59:44.797299 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.797263 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:44.797407 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:44.797334 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:44.797464 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:44.797405 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 20:59:44.797464 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:44.797461 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:46.797442843 +0000 UTC m=+39.887676002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : secret "router-metrics-certs-default" not found Jul 15 20:59:44.797572 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:44.797479 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:46.797470139 +0000 UTC m=+39.887703295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : configmap references non-existent config key: service-ca.crt Jul 15 20:59:45.770019 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:45.768942 2573 generic.go:356] "Generic (PLEG): container finished" podID="2cb2f081-1c5a-49ed-82bd-2ceb78c3be90" containerID="1e70810b87cd01b823b94a424ca82050027b26fd355198c7c64ae7e4ad45fcbe" exitCode=0 Jul 15 20:59:45.770019 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:45.769019 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" event={"ID":"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90","Type":"ContainerDied","Data":"1e70810b87cd01b823b94a424ca82050027b26fd355198c7c64ae7e4ad45fcbe"} Jul 15 20:59:46.007826 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:46.007746 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:46.007826 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:46.007815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:46.008197 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.008037 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 20:59:46.008197 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.008052 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f7469478-q45vb: secret "image-registry-tls" not found Jul 15 20:59:46.008197 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.008109 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls podName:f8a2ffcc-698c-43ae-8659-ed37b95f5fd5 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:50.008091653 +0000 UTC m=+43.098324825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls") pod "image-registry-77f7469478-q45vb" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5") : secret "image-registry-tls" not found Jul 15 20:59:46.008542 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.008517 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 20:59:46.008620 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.008588 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls podName:47f9f4d6-0555-4123-b64b-5f753c9e1160 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:50.008570737 +0000 UTC m=+43.098803910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls") pod "dns-default-jmfmx" (UID: "47f9f4d6-0555-4123-b64b-5f753c9e1160") : secret "dns-default-metrics-tls" not found Jul 15 20:59:46.713860 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:46.713816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:46.714038 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:46.714005 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:46.714038 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.714016 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:46.714146 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.714092 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls podName:84e688bb-4303-4ac4-b180-3ae21fa17823 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:50.714069703 +0000 UTC m=+43.804302890 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6pcv7" (UID: "84e688bb-4303-4ac4-b180-3ae21fa17823") : secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:46.714146 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.714140 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 20:59:46.714284 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.714210 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls podName:88ea6a7b-188f-41a2-818c-f5780e71b44f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:50.714193538 +0000 UTC m=+43.804426697 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls") pod "cluster-samples-operator-65db858894-j2w2l" (UID: "88ea6a7b-188f-41a2-818c-f5780e71b44f") : secret "samples-operator-tls" not found Jul 15 20:59:46.776344 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:46.776314 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" event={"ID":"2cb2f081-1c5a-49ed-82bd-2ceb78c3be90","Type":"ContainerStarted","Data":"8721b83bf942fce60e7ed5a2dd719e287206afaa9883c16631136cc4b61f43a3"} Jul 15 20:59:46.801344 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:46.801286 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zpdrq" podStartSLOduration=9.562264052 podStartE2EDuration="38.801265547s" totalImagesPullingTime="29.239001495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:46.800535118 +0000 UTC m=+39.890768299" watchObservedRunningTime="2026-07-15 20:59:46.801265547 +0000 UTC m=+39.891498726" Jul 15 20:59:46.814798 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:46.814772 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:46.814926 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:46.814834 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:46.814973 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.814963 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:50.814922814 +0000 UTC m=+43.905155988 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : configmap references non-existent config key: service-ca.crt Jul 15 20:59:46.815019 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.814981 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 20:59:46.815057 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:46.815034 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:50.815021908 +0000 UTC m=+43.905255063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : secret "router-metrics-certs-default" not found Jul 15 20:59:50.044742 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.044716 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:50.045206 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.044765 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:50.045206 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.044853 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 20:59:50.045206 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.044896 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 20:59:50.045206 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.044910 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f7469478-q45vb: secret "image-registry-tls" not found Jul 15 20:59:50.045206 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.044917 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls podName:47f9f4d6-0555-4123-b64b-5f753c9e1160 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:58.044900752 +0000 UTC m=+51.135133908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls") pod "dns-default-jmfmx" (UID: "47f9f4d6-0555-4123-b64b-5f753c9e1160") : secret "dns-default-metrics-tls" not found Jul 15 20:59:50.045206 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.044959 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls podName:f8a2ffcc-698c-43ae-8659-ed37b95f5fd5 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:58.044941912 +0000 UTC m=+51.135175099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls") pod "image-registry-77f7469478-q45vb" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5") : secret "image-registry-tls" not found Jul 15 20:59:50.750157 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.750115 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:50.750419 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.750254 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:50.750419 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.750308 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:50.750419 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.750343 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 20:59:50.750419 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.750388 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls podName:84e688bb-4303-4ac4-b180-3ae21fa17823 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:58.750367476 +0000 UTC m=+51.840600636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6pcv7" (UID: "84e688bb-4303-4ac4-b180-3ae21fa17823") : secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:50.750419 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.750415 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls podName:88ea6a7b-188f-41a2-818c-f5780e71b44f nodeName:}" failed. No retries permitted until 2026-07-15 20:59:58.750405807 +0000 UTC m=+51.840638967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls") pod "cluster-samples-operator-65db858894-j2w2l" (UID: "88ea6a7b-188f-41a2-818c-f5780e71b44f") : secret "samples-operator-tls" not found Jul 15 20:59:50.790203 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.790120 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp" event={"ID":"79083cd2-3480-42d0-bdf5-a5719b9f8995","Type":"ContainerStarted","Data":"220b5fb18260b0b07065e19467604e4db6dd112544bc8c679c541956f60cf783"} Jul 15 20:59:50.791796 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.791764 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" event={"ID":"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88","Type":"ContainerStarted","Data":"cd5e64922ef45574a7de0f177f24a260c7613d8f49da7616a7fb7bd5e148bab8"} Jul 15 20:59:50.793317 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.793289 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" event={"ID":"af1545f9-cf6b-4ba4-a00e-2c74949572c3","Type":"ContainerStarted","Data":"2340bf25c01e987ed0f990470ccbf1101b858237cb0ec288f64d047dbb1b8f3e"} Jul 15 20:59:50.795084 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.795059 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/0.log" Jul 15 20:59:50.795202 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.795092 2573 generic.go:356] "Generic (PLEG): container finished" podID="cb05969b-c83d-4c37-8c63-1221309d2f8b" containerID="173849cd2e8a35122313ea7ad602bb89b802853c20419a3d804beb4f57f7484a" exitCode=255 Jul 15 20:59:50.795202 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.795154 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jwz4p" event={"ID":"cb05969b-c83d-4c37-8c63-1221309d2f8b","Type":"ContainerDied","Data":"173849cd2e8a35122313ea7ad602bb89b802853c20419a3d804beb4f57f7484a"} Jul 15 20:59:50.795360 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.795344 2573 scope.go:117] "RemoveContainer" containerID="173849cd2e8a35122313ea7ad602bb89b802853c20419a3d804beb4f57f7484a" Jul 15 20:59:50.796823 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.796795 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5jlnp" event={"ID":"4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1","Type":"ContainerStarted","Data":"920bf4da68a69c23ed03ffb80813d4c46a27c937f25029e77358798a0534247c"} Jul 15 20:59:50.798389 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.798355 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" event={"ID":"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff","Type":"ContainerStarted","Data":"b1e9fc455e1479b2b7246bb354599433adde6833b5a71faca9cba73fbe4626e4"} Jul 15 20:59:50.828416 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.828236 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" podStartSLOduration=1.809338235 podStartE2EDuration="7.828221891s" totalImagesPullingTime="6.018883656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:50.828120717 +0000 UTC m=+43.918353895" watchObservedRunningTime="2026-07-15 20:59:50.828221891 +0000 UTC m=+43.918455067" Jul 15 20:59:50.828574 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.828548 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-rs6fp" podStartSLOduration=2.77505868 podStartE2EDuration="8.82854195s" totalImagesPullingTime="6.05348327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:50.808495589 +0000 UTC m=+43.898728762" watchObservedRunningTime="2026-07-15 20:59:50.82854195 +0000 UTC m=+43.918775128" Jul 15 20:59:50.849214 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.849150 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" podStartSLOduration=2.825008034 podStartE2EDuration="8.849132259s" totalImagesPullingTime="6.024124225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:50.848527732 +0000 UTC m=+43.938760917" watchObservedRunningTime="2026-07-15 20:59:50.849132259 +0000 UTC m=+43.939365438" Jul 15 20:59:50.851022 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.850996 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:50.851116 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.851060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:50.851192 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.851125 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 20:59:50.851192 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.851189 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:58.85116393 +0000 UTC m=+51.941397086 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : secret "router-metrics-certs-default" not found Jul 15 20:59:50.851315 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:50.851242 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 20:59:58.851223137 +0000 UTC m=+51.941456306 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : configmap references non-existent config key: service-ca.crt Jul 15 20:59:50.876001 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.875962 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" podStartSLOduration=2.858365784 podStartE2EDuration="8.875947845s" totalImagesPullingTime="6.017582061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:50.875724285 +0000 UTC m=+43.965957467" watchObservedRunningTime="2026-07-15 20:59:50.875947845 +0000 UTC m=+43.966181023" Jul 15 20:59:50.894876 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:50.894826 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5jlnp" podStartSLOduration=2.874292218 podStartE2EDuration="8.894808828s" totalImagesPullingTime="6.02051661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:59:50.892868916 +0000 UTC m=+43.983102096" watchObservedRunningTime="2026-07-15 20:59:50.894808828 +0000 UTC m=+43.985042007" Jul 15 20:59:51.446557 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.446527 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r8j48"] Jul 15 20:59:51.465286 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.465256 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.468496 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.468296 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 20:59:51.469184 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.469147 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m65gh\"" Jul 15 20:59:51.469390 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.469369 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r8j48"] Jul 15 20:59:51.470273 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.470212 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 20:59:51.558696 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.558615 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.558696 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.558674 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-data-volume\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.558893 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.558739 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jbz9\" (UniqueName: \"kubernetes.io/projected/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-api-access-6jbz9\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.558945 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.558900 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-crio-socket\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.558994 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.558978 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.659993 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.659956 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.660185 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.660010 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-data-volume\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.660185 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.660030 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jbz9\" (UniqueName: \"kubernetes.io/projected/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-api-access-6jbz9\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.660185 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.660103 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-crio-socket\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.660185 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:51.660132 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 20:59:51.660185 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.660163 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.660408 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:51.660218 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls podName:9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d nodeName:}" failed. No retries permitted until 2026-07-15 20:59:52.160196676 +0000 UTC m=+45.250429848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8j48" (UID: "9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d") : secret "insights-runtime-extractor-tls" not found Jul 15 20:59:51.660408 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:51.660253 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 20:59:51.660408 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:51.660302 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm podName:9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d nodeName:}" failed. No retries permitted until 2026-07-15 20:59:52.160287985 +0000 UTC m=+45.250521148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r8j48" (UID: "9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d") : configmap "kube-rbac-proxy" not found Jul 15 20:59:51.660408 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.660336 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-data-volume\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.660408 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.660373 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-crio-socket\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.670774 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.670745 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jbz9\" (UniqueName: \"kubernetes.io/projected/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-api-access-6jbz9\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:51.803476 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.803446 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 20:59:51.803845 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.803829 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/0.log" Jul 15 20:59:51.803909 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.803862 2573 generic.go:356] "Generic (PLEG): container finished" podID="cb05969b-c83d-4c37-8c63-1221309d2f8b" containerID="6f8b84eff44b4c13896d678181a8970be0d39a1ade856f08fa2e24e1a5549f6e" exitCode=255 Jul 15 20:59:51.804000 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.803979 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jwz4p" event={"ID":"cb05969b-c83d-4c37-8c63-1221309d2f8b","Type":"ContainerDied","Data":"6f8b84eff44b4c13896d678181a8970be0d39a1ade856f08fa2e24e1a5549f6e"} Jul 15 20:59:51.804056 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.804031 2573 scope.go:117] "RemoveContainer" containerID="173849cd2e8a35122313ea7ad602bb89b802853c20419a3d804beb4f57f7484a" Jul 15 20:59:51.804254 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:51.804226 2573 scope.go:117] "RemoveContainer" containerID="6f8b84eff44b4c13896d678181a8970be0d39a1ade856f08fa2e24e1a5549f6e" Jul 15 20:59:51.804450 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:51.804424 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jwz4p_openshift-console-operator(cb05969b-c83d-4c37-8c63-1221309d2f8b)\"" pod="openshift-console-operator/console-operator-7866497494-jwz4p" podUID="cb05969b-c83d-4c37-8c63-1221309d2f8b" Jul 15 20:59:52.165661 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:52.165571 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:52.165661 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:52.165633 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:52.165861 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:52.165749 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 20:59:52.165861 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:52.165773 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 20:59:52.165861 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:52.165818 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm podName:9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d nodeName:}" failed. No retries permitted until 2026-07-15 20:59:53.165797949 +0000 UTC m=+46.256031119 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r8j48" (UID: "9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d") : configmap "kube-rbac-proxy" not found Jul 15 20:59:52.165861 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:52.165834 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls podName:9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d nodeName:}" failed. No retries permitted until 2026-07-15 20:59:53.165827233 +0000 UTC m=+46.256060389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8j48" (UID: "9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d") : secret "insights-runtime-extractor-tls" not found Jul 15 20:59:52.811596 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:52.811566 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 20:59:52.811968 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:52.811851 2573 scope.go:117] "RemoveContainer" containerID="6f8b84eff44b4c13896d678181a8970be0d39a1ade856f08fa2e24e1a5549f6e" Jul 15 20:59:52.812012 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:52.811997 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jwz4p_openshift-console-operator(cb05969b-c83d-4c37-8c63-1221309d2f8b)\"" pod="openshift-console-operator/console-operator-7866497494-jwz4p" podUID="cb05969b-c83d-4c37-8c63-1221309d2f8b" Jul 15 20:59:53.174521 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:53.174435 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:53.174521 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:53.174504 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:53.174697 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:53.174547 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 20:59:53.174697 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:53.174608 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm podName:9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d nodeName:}" failed. No retries permitted until 2026-07-15 20:59:55.174594079 +0000 UTC m=+48.264827236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r8j48" (UID: "9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d") : configmap "kube-rbac-proxy" not found Jul 15 20:59:53.174697 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:53.174650 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 20:59:53.174804 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:53.174709 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls podName:9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d nodeName:}" failed. No retries permitted until 2026-07-15 20:59:55.174695999 +0000 UTC m=+48.264929158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8j48" (UID: "9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d") : secret "insights-runtime-extractor-tls" not found Jul 15 20:59:53.323379 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:53.323342 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:53.323542 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:53.323437 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 20:59:53.629001 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:53.628976 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pw8ln_17f76060-64b0-4ec1-ae6e-18eaea915761/dns-node-resolver/0.log" Jul 15 20:59:53.815569 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:53.815536 2573 scope.go:117] "RemoveContainer" containerID="6f8b84eff44b4c13896d678181a8970be0d39a1ade856f08fa2e24e1a5549f6e" Jul 15 20:59:53.816015 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:53.815728 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-jwz4p_openshift-console-operator(cb05969b-c83d-4c37-8c63-1221309d2f8b)\"" pod="openshift-console-operator/console-operator-7866497494-jwz4p" podUID="cb05969b-c83d-4c37-8c63-1221309d2f8b" Jul 15 20:59:54.230621 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:54.230595 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6pkk7_fda8f067-5b21-4d25-9771-77f72ab05e60/node-ca/0.log" Jul 15 20:59:55.028321 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:55.028292 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5jlnp_4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1/serve-healthcheck-canary/0.log" Jul 15 20:59:55.190741 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:55.190710 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:55.190892 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:55.190761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:55.190892 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:55.190857 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 20:59:55.190892 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:55.190859 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 20:59:55.190986 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:55.190909 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm podName:9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d nodeName:}" failed. No retries permitted until 2026-07-15 20:59:59.190896238 +0000 UTC m=+52.281129395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r8j48" (UID: "9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d") : configmap "kube-rbac-proxy" not found Jul 15 20:59:55.190986 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:55.190922 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls podName:9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d nodeName:}" failed. No retries permitted until 2026-07-15 20:59:59.190915783 +0000 UTC m=+52.281148938 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8j48" (UID: "9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d") : secret "insights-runtime-extractor-tls" not found Jul 15 20:59:58.116974 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:58.116938 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 20:59:58.117409 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:58.116986 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 20:59:58.117409 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.117079 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 20:59:58.117409 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.117100 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 20:59:58.117409 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.117111 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77f7469478-q45vb: secret "image-registry-tls" not found Jul 15 20:59:58.117409 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.117146 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls podName:47f9f4d6-0555-4123-b64b-5f753c9e1160 nodeName:}" failed. No retries permitted until 2026-07-15 21:00:14.117130117 +0000 UTC m=+67.207363276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls") pod "dns-default-jmfmx" (UID: "47f9f4d6-0555-4123-b64b-5f753c9e1160") : secret "dns-default-metrics-tls" not found Jul 15 20:59:58.117409 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.117159 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls podName:f8a2ffcc-698c-43ae-8659-ed37b95f5fd5 nodeName:}" failed. No retries permitted until 2026-07-15 21:00:14.117153686 +0000 UTC m=+67.207386841 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls") pod "image-registry-77f7469478-q45vb" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5") : secret "image-registry-tls" not found Jul 15 20:59:58.822280 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:58.822244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 20:59:58.822435 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.822390 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:58.822435 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:58.822431 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 20:59:58.822544 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.822451 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls podName:84e688bb-4303-4ac4-b180-3ae21fa17823 nodeName:}" failed. No retries permitted until 2026-07-15 21:00:14.822435968 +0000 UTC m=+67.912669124 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6pcv7" (UID: "84e688bb-4303-4ac4-b180-3ae21fa17823") : secret "cluster-monitoring-operator-tls" not found Jul 15 20:59:58.822544 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.822497 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 20:59:58.822615 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.822555 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls podName:88ea6a7b-188f-41a2-818c-f5780e71b44f nodeName:}" failed. No retries permitted until 2026-07-15 21:00:14.82254107 +0000 UTC m=+67.912774230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls") pod "cluster-samples-operator-65db858894-j2w2l" (UID: "88ea6a7b-188f-41a2-818c-f5780e71b44f") : secret "samples-operator-tls" not found Jul 15 20:59:58.923283 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:58.923251 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:58.923445 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:58.923302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 20:59:58.923445 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.923392 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 20:59:58.923445 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.923408 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 21:00:14.923394453 +0000 UTC m=+68.013627628 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : configmap references non-existent config key: service-ca.crt Jul 15 20:59:58.923445 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:58.923442 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs podName:14cd8b87-f21c-41c3-8cb8-7893d3f5fa28 nodeName:}" failed. No retries permitted until 2026-07-15 21:00:14.923429003 +0000 UTC m=+68.013662159 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs") pod "router-default-585bb7b99d-xcp26" (UID: "14cd8b87-f21c-41c3-8cb8-7893d3f5fa28") : secret "router-metrics-certs-default" not found Jul 15 20:59:59.226029 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:59.225945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:59.226430 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:59.226073 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 20:59:59.226430 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:59.226096 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 20:59:59.226430 ip-10-0-135-80 kubenswrapper[2573]: E0715 20:59:59.226164 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls podName:9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d nodeName:}" failed. No retries permitted until 2026-07-15 21:00:07.226141143 +0000 UTC m=+60.316374299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r8j48" (UID: "9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d") : secret "insights-runtime-extractor-tls" not found Jul 15 20:59:59.226592 ip-10-0-135-80 kubenswrapper[2573]: I0715 20:59:59.226576 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 21:00:07.297679 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:07.297648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 21:00:07.299810 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:07.299791 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r8j48\" (UID: \"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d\") " pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 21:00:07.379774 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:07.379748 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m65gh\"" Jul 15 21:00:07.387192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:07.387159 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r8j48" Jul 15 21:00:07.521292 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:07.521263 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r8j48"] Jul 15 21:00:07.524440 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:07.524412 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb747fc_0ae3_4839_b7f7_11c0a84d3d9d.slice/crio-dfff9b9b432e8ff63a517341fd319e144b17410c27f41bcb54ef925be2609981 WatchSource:0}: Error finding container dfff9b9b432e8ff63a517341fd319e144b17410c27f41bcb54ef925be2609981: Status 404 returned error can't find the container with id dfff9b9b432e8ff63a517341fd319e144b17410c27f41bcb54ef925be2609981 Jul 15 21:00:07.731116 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:07.731048 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2zh8" Jul 15 21:00:07.859533 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:07.859500 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8j48" event={"ID":"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d","Type":"ContainerStarted","Data":"dfff9b9b432e8ff63a517341fd319e144b17410c27f41bcb54ef925be2609981"} Jul 15 21:00:08.441888 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:08.441848 2573 scope.go:117] "RemoveContainer" containerID="6f8b84eff44b4c13896d678181a8970be0d39a1ade856f08fa2e24e1a5549f6e" Jul 15 21:00:08.865384 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:08.865347 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:00:08.865540 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:08.865451 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-jwz4p" event={"ID":"cb05969b-c83d-4c37-8c63-1221309d2f8b","Type":"ContainerStarted","Data":"a346be6e3acf1054f4346b914d9e214528046dacef4c6f1bf0d3e82f3c7b6931"} Jul 15 21:00:08.865839 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:08.865788 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 21:00:09.866539 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:09.866485 2573 patch_prober.go:28] interesting pod/console-operator-7866497494-jwz4p container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.12:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 21:00:09.867035 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:09.866628 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7866497494-jwz4p" podUID="cb05969b-c83d-4c37-8c63-1221309d2f8b" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.12:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 15 21:00:09.886729 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:09.886702 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-jwz4p" Jul 15 21:00:09.916559 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:09.916502 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-jwz4p" podStartSLOduration=21.897487203 podStartE2EDuration="27.916478982s" totalImagesPullingTime="6.018991779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:08.89051614 +0000 UTC m=+61.980749319" watchObservedRunningTime="2026-07-15 21:00:09.916478982 +0000 UTC m=+63.006712159" Jul 15 21:00:12.881399 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:12.881298 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8j48" event={"ID":"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d","Type":"ContainerStarted","Data":"112e30d781df1014f4b8641ea6a373b93d6481cce0f565b0649d68ff11c749da"} Jul 15 21:00:13.255012 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.254975 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 21:00:13.255433 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.255024 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 21:00:13.255433 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.255077 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 21:00:13.259761 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.259630 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 21:00:13.260962 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.260793 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 21:00:13.263370 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.263350 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 21:00:13.269128 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.268845 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01016934-00c3-4dae-8db6-a118e7c3cbe9-original-pull-secret\") pod \"global-pull-secret-syncer-98h25\" (UID: \"01016934-00c3-4dae-8db6-a118e7c3cbe9\") " pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 21:00:13.269128 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.268962 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 21:00:13.269937 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.269913 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/343ee524-4530-4307-b119-1c4fb610db6f-metrics-certs\") pod \"network-metrics-daemon-kgvtw\" (UID: \"343ee524-4530-4307-b119-1c4fb610db6f\") " pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 21:00:13.279468 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.279443 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv5fg\" (UniqueName: \"kubernetes.io/projected/f37e3221-e7d4-486e-8f9c-5c0d83e702dd-kube-api-access-gv5fg\") pod \"network-check-target-z7dvf\" (UID: \"f37e3221-e7d4-486e-8f9c-5c0d83e702dd\") " pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 21:00:13.354646 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.354606 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jnsrt\"" Jul 15 21:00:13.361912 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.361829 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kgvtw" Jul 15 21:00:13.362290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.362261 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ffp72\"" Jul 15 21:00:13.363576 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.363554 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-98h25" Jul 15 21:00:13.368568 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.368401 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 21:00:13.540704 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.540624 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kgvtw"] Jul 15 21:00:13.545155 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:13.545127 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343ee524_4530_4307_b119_1c4fb610db6f.slice/crio-cd436c9ebc23fba48d7120f6b0fb5b152956a101e166cdcb6c92203c1beb895c WatchSource:0}: Error finding container cd436c9ebc23fba48d7120f6b0fb5b152956a101e166cdcb6c92203c1beb895c: Status 404 returned error can't find the container with id cd436c9ebc23fba48d7120f6b0fb5b152956a101e166cdcb6c92203c1beb895c Jul 15 21:00:13.781272 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.781183 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-98h25"] Jul 15 21:00:13.782434 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.782414 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7dvf"] Jul 15 21:00:13.785060 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:13.785023 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01016934_00c3_4dae_8db6_a118e7c3cbe9.slice/crio-8de10507844ee8c6775f0231fa2eb0356894b9e7db3fa6d52bae7d0c4575f073 WatchSource:0}: Error finding container 8de10507844ee8c6775f0231fa2eb0356894b9e7db3fa6d52bae7d0c4575f073: Status 404 returned error can't find the container with id 8de10507844ee8c6775f0231fa2eb0356894b9e7db3fa6d52bae7d0c4575f073 Jul 15 21:00:13.785926 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:13.785879 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf37e3221_e7d4_486e_8f9c_5c0d83e702dd.slice/crio-9292a169142cad44ee379883b893d4f915563eb9d7413bc2b26360ccde2cb5a6 WatchSource:0}: Error finding container 9292a169142cad44ee379883b893d4f915563eb9d7413bc2b26360ccde2cb5a6: Status 404 returned error can't find the container with id 9292a169142cad44ee379883b893d4f915563eb9d7413bc2b26360ccde2cb5a6 Jul 15 21:00:13.886530 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.886443 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kgvtw" event={"ID":"343ee524-4530-4307-b119-1c4fb610db6f","Type":"ContainerStarted","Data":"cd436c9ebc23fba48d7120f6b0fb5b152956a101e166cdcb6c92203c1beb895c"} Jul 15 21:00:13.887765 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.887734 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7dvf" event={"ID":"f37e3221-e7d4-486e-8f9c-5c0d83e702dd","Type":"ContainerStarted","Data":"9292a169142cad44ee379883b893d4f915563eb9d7413bc2b26360ccde2cb5a6"} Jul 15 21:00:13.889017 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:13.888988 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-98h25" event={"ID":"01016934-00c3-4dae-8db6-a118e7c3cbe9","Type":"ContainerStarted","Data":"8de10507844ee8c6775f0231fa2eb0356894b9e7db3fa6d52bae7d0c4575f073"} Jul 15 21:00:14.166341 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.166251 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 21:00:14.166341 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.166322 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 21:00:14.169251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.169202 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f9f4d6-0555-4123-b64b-5f753c9e1160-metrics-tls\") pod \"dns-default-jmfmx\" (UID: \"47f9f4d6-0555-4123-b64b-5f753c9e1160\") " pod="openshift-dns/dns-default-jmfmx" Jul 15 21:00:14.169590 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.169564 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"image-registry-77f7469478-q45vb\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 21:00:14.349697 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.349663 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pndn8\"" Jul 15 21:00:14.356829 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.356783 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 21:00:14.368600 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.368573 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9rgfh\"" Jul 15 21:00:14.376098 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.376077 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jmfmx" Jul 15 21:00:14.872921 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.872470 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 21:00:14.872921 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.872577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 21:00:14.875408 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.875386 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/84e688bb-4303-4ac4-b180-3ae21fa17823-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6pcv7\" (UID: \"84e688bb-4303-4ac4-b180-3ae21fa17823\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 21:00:14.876114 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.876083 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/88ea6a7b-188f-41a2-818c-f5780e71b44f-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-j2w2l\" (UID: \"88ea6a7b-188f-41a2-818c-f5780e71b44f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 21:00:14.973181 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.973123 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 21:00:14.973663 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.973211 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 21:00:14.974084 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.974030 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-service-ca-bundle\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 21:00:14.977058 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:14.976803 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14cd8b87-f21c-41c3-8cb8-7893d3f5fa28-metrics-certs\") pod \"router-default-585bb7b99d-xcp26\" (UID: \"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28\") " pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 21:00:15.111765 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:15.111728 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k9ll8\"" Jul 15 21:00:15.117023 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:15.116995 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" Jul 15 21:00:15.133423 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:15.133301 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5swh9\"" Jul 15 21:00:15.139476 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:15.139448 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" Jul 15 21:00:15.207673 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:15.207639 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-l249s\"" Jul 15 21:00:15.216032 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:15.216004 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 21:00:19.384504 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:19.384477 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jmfmx"] Jul 15 21:00:20.547644 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.547533 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-585bb7b99d-xcp26"] Jul 15 21:00:20.550684 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.550661 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7"] Jul 15 21:00:20.563389 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.563361 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l"] Jul 15 21:00:20.566588 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.566560 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77f7469478-q45vb"] Jul 15 21:00:20.681837 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:20.681800 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14cd8b87_f21c_41c3_8cb8_7893d3f5fa28.slice/crio-6dd425af20977d43956a8fab0dcaccc0e19c1449382025d5a15ab496980b0d4a WatchSource:0}: Error finding container 6dd425af20977d43956a8fab0dcaccc0e19c1449382025d5a15ab496980b0d4a: Status 404 returned error can't find the container with id 6dd425af20977d43956a8fab0dcaccc0e19c1449382025d5a15ab496980b0d4a Jul 15 21:00:20.682248 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:20.682222 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e688bb_4303_4ac4_b180_3ae21fa17823.slice/crio-c14a7567a4eb03ce97a35b1af0139f6bde8a92139eb53bdf92d54d75038694f8 WatchSource:0}: Error finding container c14a7567a4eb03ce97a35b1af0139f6bde8a92139eb53bdf92d54d75038694f8: Status 404 returned error can't find the container with id c14a7567a4eb03ce97a35b1af0139f6bde8a92139eb53bdf92d54d75038694f8 Jul 15 21:00:20.683081 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:20.683054 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a2ffcc_698c_43ae_8659_ed37b95f5fd5.slice/crio-81748294cb5a874c38320a6c36ae113ede2598fabbbb6ee9785bfff2ddee38a3 WatchSource:0}: Error finding container 81748294cb5a874c38320a6c36ae113ede2598fabbbb6ee9785bfff2ddee38a3: Status 404 returned error can't find the container with id 81748294cb5a874c38320a6c36ae113ede2598fabbbb6ee9785bfff2ddee38a3 Jul 15 21:00:20.921571 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.921486 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jmfmx" event={"ID":"47f9f4d6-0555-4123-b64b-5f753c9e1160","Type":"ContainerStarted","Data":"be7ffd097bacd998e9f4a1cc255278cd28a9f5ff4ad9a2b4cbe1456bab8602f3"} Jul 15 21:00:20.923897 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.923835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kgvtw" event={"ID":"343ee524-4530-4307-b119-1c4fb610db6f","Type":"ContainerStarted","Data":"1266d128b368c48ad52698ce76454da268b54dc414ce9959060e87556901e258"} Jul 15 21:00:20.923897 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.923893 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kgvtw" event={"ID":"343ee524-4530-4307-b119-1c4fb610db6f","Type":"ContainerStarted","Data":"92063fa2eadb11123900ef372815857736ce113d1e2447f677288c0fd4937ab6"} Jul 15 21:00:20.926051 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.926001 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7dvf" event={"ID":"f37e3221-e7d4-486e-8f9c-5c0d83e702dd","Type":"ContainerStarted","Data":"49baa3b370f5d41be830eb325ceb9ac00530c51e5298fceb3edb8edfc7d28b51"} Jul 15 21:00:20.926216 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.926073 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 21:00:20.927827 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.927792 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" event={"ID":"88ea6a7b-188f-41a2-818c-f5780e71b44f","Type":"ContainerStarted","Data":"0ebc320aa9e9915131b648c7b89383e17ee83b7b91f34681ad9ff1cd58f7a2a0"} Jul 15 21:00:20.930916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.930891 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8j48" event={"ID":"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d","Type":"ContainerStarted","Data":"b77195d04fe6def8736a459427a0297397fc07ed893d11157700e71d0253162b"} Jul 15 21:00:20.932706 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.932643 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77f7469478-q45vb" event={"ID":"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5","Type":"ContainerStarted","Data":"5a8b8f715d4c44e64d04f5beaaa33e25a992d848f1e1ddeaa38ac62d1ed95575"} Jul 15 21:00:20.932706 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.932680 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77f7469478-q45vb" event={"ID":"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5","Type":"ContainerStarted","Data":"81748294cb5a874c38320a6c36ae113ede2598fabbbb6ee9785bfff2ddee38a3"} Jul 15 21:00:20.932879 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.932763 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 21:00:20.934675 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.934619 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-585bb7b99d-xcp26" event={"ID":"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28","Type":"ContainerStarted","Data":"7c88f6397628db75400d6d6609b474667eab133771c1e3ff62de04de925fef56"} Jul 15 21:00:20.934675 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.934656 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-585bb7b99d-xcp26" event={"ID":"14cd8b87-f21c-41c3-8cb8-7893d3f5fa28","Type":"ContainerStarted","Data":"6dd425af20977d43956a8fab0dcaccc0e19c1449382025d5a15ab496980b0d4a"} Jul 15 21:00:20.937117 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.937096 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" event={"ID":"84e688bb-4303-4ac4-b180-3ae21fa17823","Type":"ContainerStarted","Data":"c14a7567a4eb03ce97a35b1af0139f6bde8a92139eb53bdf92d54d75038694f8"} Jul 15 21:00:20.939013 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.938965 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-98h25" event={"ID":"01016934-00c3-4dae-8db6-a118e7c3cbe9","Type":"ContainerStarted","Data":"4d07edf476cff1cf47329e999ff0d012a765dd0fce8862d8f83ef5bd7dacdbe0"} Jul 15 21:00:20.956809 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.956748 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kgvtw" podStartSLOduration=67.264916685 podStartE2EDuration="1m12.956730875s" totalImagesPullingTime="5.69181419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:20.955651374 +0000 UTC m=+74.045884544" watchObservedRunningTime="2026-07-15 21:00:20.956730875 +0000 UTC m=+74.046964055" Jul 15 21:00:20.988532 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:20.988458 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-585bb7b99d-xcp26" podStartSLOduration=37.98843846 podStartE2EDuration="37.98843846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:20.986192061 +0000 UTC m=+74.076425241" watchObservedRunningTime="2026-07-15 21:00:20.98843846 +0000 UTC m=+74.078671637" Jul 15 21:00:21.019949 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:21.019850 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z7dvf" podStartSLOduration=66.08484916 podStartE2EDuration="1m13.019827856s" totalImagesPullingTime="6.934978696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:21.018722137 +0000 UTC m=+74.108955314" watchObservedRunningTime="2026-07-15 21:00:21.019827856 +0000 UTC m=+74.110061036" Jul 15 21:00:21.042873 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:21.042779 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-98h25" podStartSLOduration=66.530119469 podStartE2EDuration="1m13.042758099s" totalImagesPullingTime="6.51263863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:21.041406603 +0000 UTC m=+74.131639783" watchObservedRunningTime="2026-07-15 21:00:21.042758099 +0000 UTC m=+74.132991279" Jul 15 21:00:21.072616 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:21.072521 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77f7469478-q45vb" podStartSLOduration=73.072493205 podStartE2EDuration="1m13.072493205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:21.071653065 +0000 UTC m=+74.161886240" watchObservedRunningTime="2026-07-15 21:00:21.072493205 +0000 UTC m=+74.162726384" Jul 15 21:00:21.216576 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:21.216530 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 21:00:21.219602 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:21.219578 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 21:00:21.946107 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:21.946058 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 21:00:21.948039 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:21.947853 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-585bb7b99d-xcp26" Jul 15 21:00:25.960918 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:25.960862 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" event={"ID":"88ea6a7b-188f-41a2-818c-f5780e71b44f","Type":"ContainerStarted","Data":"033f810b1283f9e26a8298a98e193f55fb3dbe4e4ad695dc76f7b245f5b07b26"} Jul 15 21:00:25.961389 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:25.960973 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" event={"ID":"88ea6a7b-188f-41a2-818c-f5780e71b44f","Type":"ContainerStarted","Data":"5cfd7e66ecfadecc65e2a6b9a5cb3af6762073dafaa29ad41a1324da15ff41df"} Jul 15 21:00:25.962869 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:25.962845 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r8j48" event={"ID":"9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d","Type":"ContainerStarted","Data":"f87eeccef9ba4bd6fc8c7e592a7501d5b5d33b5e3ee688c07f5be4420763b760"} Jul 15 21:00:25.964236 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:25.964210 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" event={"ID":"84e688bb-4303-4ac4-b180-3ae21fa17823","Type":"ContainerStarted","Data":"367fc98e08f72c4ed66cb95b08f0f9c9e12d9a571c79462c300bdda6238795c5"} Jul 15 21:00:25.965730 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:25.965710 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jmfmx" event={"ID":"47f9f4d6-0555-4123-b64b-5f753c9e1160","Type":"ContainerStarted","Data":"85ecef88d24ed3fc1b811b1c52c5a83290372e94b643be58371a7d0a88cb6fe9"} Jul 15 21:00:25.965868 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:25.965746 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jmfmx" event={"ID":"47f9f4d6-0555-4123-b64b-5f753c9e1160","Type":"ContainerStarted","Data":"b651494742640f151978f08549c0b615638616c814300cf42a333b980b5b16af"} Jul 15 21:00:25.965868 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:25.965782 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jmfmx" Jul 15 21:00:26.014673 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:26.014630 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-j2w2l" podStartSLOduration=39.624674643 podStartE2EDuration="44.014616433s" totalImagesPullingTime="4.38994179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:26.013987035 +0000 UTC m=+79.104220212" watchObservedRunningTime="2026-07-15 21:00:26.014616433 +0000 UTC m=+79.104849611" Jul 15 21:00:26.061004 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:26.060955 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jmfmx" podStartSLOduration=39.228591877 podStartE2EDuration="44.060940175s" totalImagesPullingTime="4.832348298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:26.060410764 +0000 UTC m=+79.150643942" watchObservedRunningTime="2026-07-15 21:00:26.060940175 +0000 UTC m=+79.151173352" Jul 15 21:00:26.113627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:26.113579 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r8j48" podStartSLOduration=17.775790605 podStartE2EDuration="35.113564179s" totalImagesPullingTime="17.337773574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:59:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:26.113000765 +0000 UTC m=+79.203233962" watchObservedRunningTime="2026-07-15 21:00:26.113564179 +0000 UTC m=+79.203797356" Jul 15 21:00:26.156027 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:26.155973 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6pcv7" podStartSLOduration=39.731897042 podStartE2EDuration="44.155958112s" totalImagesPullingTime="4.42406107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:26.15511436 +0000 UTC m=+79.245347553" watchObservedRunningTime="2026-07-15 21:00:26.155958112 +0000 UTC m=+79.246191342" Jul 15 21:00:35.973128 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:35.973017 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jmfmx" Jul 15 21:00:37.983421 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:37.983387 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77f7469478-q45vb"] Jul 15 21:00:37.987499 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:37.987477 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 21:00:44.330470 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.330442 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5"] Jul 15 21:00:44.424396 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.424365 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5"] Jul 15 21:00:44.424551 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.424411 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-nl9x8"] Jul 15 21:00:44.424551 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.424483 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.427240 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.427218 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-555v4\"" Jul 15 21:00:44.427555 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.427522 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 21:00:44.427555 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.427550 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 21:00:44.428272 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.428254 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 21:00:44.453649 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.453631 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4ltr5"] Jul 15 21:00:44.453757 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.453742 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.456550 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.456534 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-f4mdt\"" Jul 15 21:00:44.457380 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.457360 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 15 21:00:44.457465 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.457389 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 15 21:00:44.457942 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.457927 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 15 21:00:44.472724 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.472706 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-nl9x8"] Jul 15 21:00:44.472802 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.472790 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.475989 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.475971 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 21:00:44.476182 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.476160 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 21:00:44.476275 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.476187 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 21:00:44.476327 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.476276 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dwmzk\"" Jul 15 21:00:44.597546 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597482 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8422f2da-72a1-4ad1-af11-7b9bcd44db03-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.597546 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597512 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.597546 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597534 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-tls\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.597755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597554 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.597755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597573 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.597755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597617 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bjx2\" (UniqueName: \"kubernetes.io/projected/8422f2da-72a1-4ad1-af11-7b9bcd44db03-kube-api-access-2bjx2\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.597755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597649 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3a49d89f-adde-46c4-add4-968cc8e595ea-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.597755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1270fd96-fdce-4907-858d-8047350e1ee5-root\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.597755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1270fd96-fdce-4907-858d-8047350e1ee5-sys\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.597755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597723 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.597755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597747 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.598004 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597762 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-textfile\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.598004 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597777 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gzcc\" (UniqueName: \"kubernetes.io/projected/1270fd96-fdce-4907-858d-8047350e1ee5-kube-api-access-7gzcc\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.598004 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597794 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8422f2da-72a1-4ad1-af11-7b9bcd44db03-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.598004 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597844 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a49d89f-adde-46c4-add4-968cc8e595ea-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.598004 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597873 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8422f2da-72a1-4ad1-af11-7b9bcd44db03-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.598004 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597906 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gc4g\" (UniqueName: \"kubernetes.io/projected/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-api-access-6gc4g\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.598004 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597924 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-wtmp\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.598004 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.597957 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1270fd96-fdce-4907-858d-8047350e1ee5-metrics-client-ca\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.698808 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.698781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1270fd96-fdce-4907-858d-8047350e1ee5-metrics-client-ca\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.698942 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.698817 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8422f2da-72a1-4ad1-af11-7b9bcd44db03-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.698942 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.698838 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.698942 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.698897 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-tls\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.698942 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.698932 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.699135 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.698954 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.699135 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.698975 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bjx2\" (UniqueName: \"kubernetes.io/projected/8422f2da-72a1-4ad1-af11-7b9bcd44db03-kube-api-access-2bjx2\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.699135 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699003 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3a49d89f-adde-46c4-add4-968cc8e595ea-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.699135 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:00:44.699032 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 15 21:00:44.699135 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699076 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1270fd96-fdce-4907-858d-8047350e1ee5-root\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.699135 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:00:44.699130 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-tls podName:1270fd96-fdce-4907-858d-8047350e1ee5 nodeName:}" failed. No retries permitted until 2026-07-15 21:00:45.199106519 +0000 UTC m=+98.289339702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-tls") pod "node-exporter-4ltr5" (UID: "1270fd96-fdce-4907-858d-8047350e1ee5") : secret "node-exporter-tls" not found Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699033 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1270fd96-fdce-4907-858d-8047350e1ee5-root\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699203 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1270fd96-fdce-4907-858d-8047350e1ee5-sys\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699237 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699280 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-textfile\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699331 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gzcc\" (UniqueName: \"kubernetes.io/projected/1270fd96-fdce-4907-858d-8047350e1ee5-kube-api-access-7gzcc\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699350 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1270fd96-fdce-4907-858d-8047350e1ee5-sys\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8422f2da-72a1-4ad1-af11-7b9bcd44db03-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699401 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a49d89f-adde-46c4-add4-968cc8e595ea-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699431 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8422f2da-72a1-4ad1-af11-7b9bcd44db03-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699443 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3a49d89f-adde-46c4-add4-968cc8e595ea-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.699469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699459 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gc4g\" (UniqueName: \"kubernetes.io/projected/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-api-access-6gc4g\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.700019 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699495 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-wtmp\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.700019 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699513 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1270fd96-fdce-4907-858d-8047350e1ee5-metrics-client-ca\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.700019 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699693 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-wtmp\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.700019 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.699776 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-textfile\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.700257 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.700233 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-accelerators-collector-config\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.700367 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.700271 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a49d89f-adde-46c4-add4-968cc8e595ea-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.700672 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.700654 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8422f2da-72a1-4ad1-af11-7b9bcd44db03-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.701324 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.701299 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.701913 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.701890 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.701997 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.701898 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8422f2da-72a1-4ad1-af11-7b9bcd44db03-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.702115 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.702096 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.702449 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.702430 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8422f2da-72a1-4ad1-af11-7b9bcd44db03-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.703271 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.703249 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.711297 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.711274 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gc4g\" (UniqueName: \"kubernetes.io/projected/3a49d89f-adde-46c4-add4-968cc8e595ea-kube-api-access-6gc4g\") pod \"kube-state-metrics-785778ff74-nl9x8\" (UID: \"3a49d89f-adde-46c4-add4-968cc8e595ea\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.718519 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.718493 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gzcc\" (UniqueName: \"kubernetes.io/projected/1270fd96-fdce-4907-858d-8047350e1ee5-kube-api-access-7gzcc\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:44.721849 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.721823 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bjx2\" (UniqueName: \"kubernetes.io/projected/8422f2da-72a1-4ad1-af11-7b9bcd44db03-kube-api-access-2bjx2\") pod \"openshift-state-metrics-84bd68b896-mcpn5\" (UID: \"8422f2da-72a1-4ad1-af11-7b9bcd44db03\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.733159 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.733142 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" Jul 15 21:00:44.762435 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.762412 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" Jul 15 21:00:44.923485 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.923450 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5"] Jul 15 21:00:44.926403 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:44.926373 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8422f2da_72a1_4ad1_af11_7b9bcd44db03.slice/crio-bc8d54927d16e81e0af9213218c4db27593df0d4400d875170e8bb5f6de59621 WatchSource:0}: Error finding container bc8d54927d16e81e0af9213218c4db27593df0d4400d875170e8bb5f6de59621: Status 404 returned error can't find the container with id bc8d54927d16e81e0af9213218c4db27593df0d4400d875170e8bb5f6de59621 Jul 15 21:00:44.941781 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:44.941757 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-nl9x8"] Jul 15 21:00:44.944948 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:44.944922 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a49d89f_adde_46c4_add4_968cc8e595ea.slice/crio-548226cd949495b9ec4ffafcccd509f2029e5eef161942020b0939a74e5ae9c3 WatchSource:0}: Error finding container 548226cd949495b9ec4ffafcccd509f2029e5eef161942020b0939a74e5ae9c3: Status 404 returned error can't find the container with id 548226cd949495b9ec4ffafcccd509f2029e5eef161942020b0939a74e5ae9c3 Jul 15 21:00:45.040537 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.040505 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" event={"ID":"3a49d89f-adde-46c4-add4-968cc8e595ea","Type":"ContainerStarted","Data":"548226cd949495b9ec4ffafcccd509f2029e5eef161942020b0939a74e5ae9c3"} Jul 15 21:00:45.042150 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.042127 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" event={"ID":"8422f2da-72a1-4ad1-af11-7b9bcd44db03","Type":"ContainerStarted","Data":"952e7dce6f35b4ad512180b32809ed985b0333b7423e430891e352a1ffcb647e"} Jul 15 21:00:45.042270 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.042186 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" event={"ID":"8422f2da-72a1-4ad1-af11-7b9bcd44db03","Type":"ContainerStarted","Data":"bc8d54927d16e81e0af9213218c4db27593df0d4400d875170e8bb5f6de59621"} Jul 15 21:00:45.204738 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.204699 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-tls\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:45.204908 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:00:45.204845 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 15 21:00:45.204971 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:00:45.204917 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-tls podName:1270fd96-fdce-4907-858d-8047350e1ee5 nodeName:}" failed. No retries permitted until 2026-07-15 21:00:46.204896617 +0000 UTC m=+99.295129776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-tls") pod "node-exporter-4ltr5" (UID: "1270fd96-fdce-4907-858d-8047350e1ee5") : secret "node-exporter-tls" not found Jul 15 21:00:45.436645 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.436561 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 21:00:45.452248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.452218 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.456513 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.456489 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 21:00:45.456650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.456510 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 21:00:45.456650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.456494 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 21:00:45.456802 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.456789 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 21:00:45.456970 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.456954 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 21:00:45.457057 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.456970 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 21:00:45.457057 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.456984 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vzx5z\"" Jul 15 21:00:45.457185 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.457079 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 21:00:45.457339 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.457320 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 21:00:45.459602 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.459584 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 21:00:45.463354 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.463334 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 21:00:45.607732 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.607701 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.607904 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.607756 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-config-out\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.607904 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.607794 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.607904 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.607823 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.607904 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.607850 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.607904 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.607875 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.608208 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.607948 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.608208 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.607989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkqtn\" (UniqueName: \"kubernetes.io/projected/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-kube-api-access-fkqtn\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.608208 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.608037 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.608208 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.608063 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.608208 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.608093 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.608208 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.608157 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-web-config\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.608469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.608227 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709154 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709220 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709266 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-config-out\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709288 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709313 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709339 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709400 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709429 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkqtn\" (UniqueName: \"kubernetes.io/projected/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-kube-api-access-fkqtn\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709499 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709529 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.709585 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-web-config\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.713916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.711841 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.715236 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.712373 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.715236 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.713529 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-web-config\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.715236 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.713781 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.715236 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.714322 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.715902 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.715823 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.715902 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.715825 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.716885 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.716861 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.720826 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.720627 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.720826 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.720794 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.721431 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.721004 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.721765 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.721728 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-config-out\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.725865 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.725825 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkqtn\" (UniqueName: \"kubernetes.io/projected/d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7-kube-api-access-fkqtn\") pod \"alertmanager-main-0\" (UID: \"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.762163 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.762137 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 21:00:45.953431 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:45.953398 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 21:00:45.956940 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:45.956906 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c9d1f6_b33d_455c_97f7_ce5d5736a2d7.slice/crio-88e647a08a69bcb427bb73662805e80bfc2a239f33e0962ce65185bfd1226d2b WatchSource:0}: Error finding container 88e647a08a69bcb427bb73662805e80bfc2a239f33e0962ce65185bfd1226d2b: Status 404 returned error can't find the container with id 88e647a08a69bcb427bb73662805e80bfc2a239f33e0962ce65185bfd1226d2b Jul 15 21:00:46.050695 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:46.050665 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7","Type":"ContainerStarted","Data":"88e647a08a69bcb427bb73662805e80bfc2a239f33e0962ce65185bfd1226d2b"} Jul 15 21:00:46.053673 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:46.053629 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" event={"ID":"8422f2da-72a1-4ad1-af11-7b9bcd44db03","Type":"ContainerStarted","Data":"222a94686554fd67a9a85076dbe5c9f56e6d43e8f9a5debec29d19b8c5b536fa"} Jul 15 21:00:46.215208 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:46.215145 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-tls\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:46.217864 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:46.217837 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1270fd96-fdce-4907-858d-8047350e1ee5-node-exporter-tls\") pod \"node-exporter-4ltr5\" (UID: \"1270fd96-fdce-4907-858d-8047350e1ee5\") " pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:46.281397 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:46.281368 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4ltr5" Jul 15 21:00:46.925198 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:46.925151 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1270fd96_fdce_4907_858d_8047350e1ee5.slice/crio-f123b6f54aa886709b47189075ba2ae02386917095a1251b36e445782ad07db0 WatchSource:0}: Error finding container f123b6f54aa886709b47189075ba2ae02386917095a1251b36e445782ad07db0: Status 404 returned error can't find the container with id f123b6f54aa886709b47189075ba2ae02386917095a1251b36e445782ad07db0 Jul 15 21:00:47.059088 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:47.059054 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4ltr5" event={"ID":"1270fd96-fdce-4907-858d-8047350e1ee5","Type":"ContainerStarted","Data":"f123b6f54aa886709b47189075ba2ae02386917095a1251b36e445782ad07db0"} Jul 15 21:00:47.061136 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:47.061108 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" event={"ID":"3a49d89f-adde-46c4-add4-968cc8e595ea","Type":"ContainerStarted","Data":"6cd085cfe41d02ed358dfbe7db34f372459efc50303c8c14197528a387539aaa"} Jul 15 21:00:47.063716 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:47.063663 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" event={"ID":"8422f2da-72a1-4ad1-af11-7b9bcd44db03","Type":"ContainerStarted","Data":"65c102fa7c42ac9e5a795fe6c246028dcf7aea9701b4b8216153d3cf66cd1793"} Jul 15 21:00:48.071117 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.070936 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" event={"ID":"3a49d89f-adde-46c4-add4-968cc8e595ea","Type":"ContainerStarted","Data":"6d5423ec04ed0a22cfdc882987aca92aaa505346086401faf55d54bf3bd2ec90"} Jul 15 21:00:48.071117 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.071077 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" event={"ID":"3a49d89f-adde-46c4-add4-968cc8e595ea","Type":"ContainerStarted","Data":"f19e08a82ffc0f82a2df97232b45fb40adce9606f72e15965c350810fcf34063"} Jul 15 21:00:48.072659 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.072629 2573 generic.go:356] "Generic (PLEG): container finished" podID="d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7" containerID="d2f7c76cf68e77a627e6a72aef2f8e9f6d0f5fe22f5756b04c0d6ec9ad0af800" exitCode=0 Jul 15 21:00:48.072795 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.072773 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7","Type":"ContainerDied","Data":"d2f7c76cf68e77a627e6a72aef2f8e9f6d0f5fe22f5756b04c0d6ec9ad0af800"} Jul 15 21:00:48.093610 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.093561 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-nl9x8" podStartSLOduration=2.116462168 podStartE2EDuration="4.09354504s" totalImagesPullingTime="1.977082872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:48.092396233 +0000 UTC m=+101.182629436" watchObservedRunningTime="2026-07-15 21:00:48.09354504 +0000 UTC m=+101.183778219" Jul 15 21:00:48.120675 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.120622 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcpn5" podStartSLOduration=2.33350933 podStartE2EDuration="4.120602521s" totalImagesPullingTime="1.787093191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:48.120114703 +0000 UTC m=+101.210347897" watchObservedRunningTime="2026-07-15 21:00:48.120602521 +0000 UTC m=+101.210835702" Jul 15 21:00:48.926080 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.926005 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b6d894486-d76t9"] Jul 15 21:00:48.942134 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.941478 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:48.944233 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.944203 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b6d894486-d76t9"] Jul 15 21:00:48.959459 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.959409 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-782q8\"" Jul 15 21:00:48.959630 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.959613 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 15 21:00:48.959706 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.959670 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cu4bjgmc26v6q\"" Jul 15 21:00:48.960826 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.960769 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 15 21:00:48.961046 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.961029 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 15 21:00:48.961928 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:48.961908 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 15 21:00:49.043151 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.043106 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-secret-metrics-server-tls\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.043327 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.043215 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-secret-metrics-server-client-certs\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.043327 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.043269 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkwtn\" (UniqueName: \"kubernetes.io/projected/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-kube-api-access-fkwtn\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.043464 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.043447 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.043566 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.043533 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-audit-log\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.043748 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.043710 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-client-ca-bundle\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.043893 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.043814 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-metrics-server-audit-profiles\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.081315 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.081091 2573 generic.go:356] "Generic (PLEG): container finished" podID="1270fd96-fdce-4907-858d-8047350e1ee5" containerID="4a411a61a27cb47fb49a52c254b2fc3c1dba7edbdcfdf93567bd090ac55c42af" exitCode=0 Jul 15 21:00:49.081315 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.081142 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4ltr5" event={"ID":"1270fd96-fdce-4907-858d-8047350e1ee5","Type":"ContainerDied","Data":"4a411a61a27cb47fb49a52c254b2fc3c1dba7edbdcfdf93567bd090ac55c42af"} Jul 15 21:00:49.146201 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.144834 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-metrics-server-audit-profiles\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.146201 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.144914 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-secret-metrics-server-tls\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.146201 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.144952 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-secret-metrics-server-client-certs\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.146201 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.145000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwtn\" (UniqueName: \"kubernetes.io/projected/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-kube-api-access-fkwtn\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.146201 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.145047 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.146201 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.145083 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-audit-log\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.146201 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.145107 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-client-ca-bundle\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.146662 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.146528 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4"] Jul 15 21:00:49.146738 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.146705 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-audit-log\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.146944 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.146922 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.147221 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.147198 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-metrics-server-audit-profiles\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.147967 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.147949 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-client-ca-bundle\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.148325 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.148303 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-secret-metrics-server-tls\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.148413 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.148381 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-secret-metrics-server-client-certs\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.169318 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.168078 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkwtn\" (UniqueName: \"kubernetes.io/projected/cb912f3a-ca31-4e5a-ae84-905f9198d5bc-kube-api-access-fkwtn\") pod \"metrics-server-5b6d894486-d76t9\" (UID: \"cb912f3a-ca31-4e5a-ae84-905f9198d5bc\") " pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.177899 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.177876 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4"] Jul 15 21:00:49.178035 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.178015 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:49.180991 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.180970 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 15 21:00:49.181335 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.181314 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dzwqd\"" Jul 15 21:00:49.255155 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.255128 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:00:49.347370 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.347332 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n22zs\" (UniqueName: \"kubernetes.io/projected/3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9-kube-api-access-n22zs\") pod \"monitoring-plugin-5f68df7f58-jlpb4\" (UID: \"3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:49.347546 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.347429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-jlpb4\" (UID: \"3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:49.422376 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.422342 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b6d894486-d76t9"] Jul 15 21:00:49.448606 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.448526 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-jlpb4\" (UID: \"3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:49.448714 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.448626 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n22zs\" (UniqueName: \"kubernetes.io/projected/3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9-kube-api-access-n22zs\") pod \"monitoring-plugin-5f68df7f58-jlpb4\" (UID: \"3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:49.451455 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.451426 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-jlpb4\" (UID: \"3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:49.462307 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.462278 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n22zs\" (UniqueName: \"kubernetes.io/projected/3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9-kube-api-access-n22zs\") pod \"monitoring-plugin-5f68df7f58-jlpb4\" (UID: \"3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:49.495554 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.495521 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:49.744944 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:49.744881 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb912f3a_ca31_4e5a_ae84_905f9198d5bc.slice/crio-2977e77fed234ed7ef115fc2642c601579a53ed06eeccb69d63c97b6fe434b3b WatchSource:0}: Error finding container 2977e77fed234ed7ef115fc2642c601579a53ed06eeccb69d63c97b6fe434b3b: Status 404 returned error can't find the container with id 2977e77fed234ed7ef115fc2642c601579a53ed06eeccb69d63c97b6fe434b3b Jul 15 21:00:49.865352 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:49.865325 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4"] Jul 15 21:00:49.867645 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:00:49.867620 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8d56aa_8bc0_4f5e_8acb_697b9a6b8cc9.slice/crio-064afe380613fac49b5d124f8cdbec66156777fb7a7a8a5b1d23b740d2d3c938 WatchSource:0}: Error finding container 064afe380613fac49b5d124f8cdbec66156777fb7a7a8a5b1d23b740d2d3c938: Status 404 returned error can't find the container with id 064afe380613fac49b5d124f8cdbec66156777fb7a7a8a5b1d23b740d2d3c938 Jul 15 21:00:50.085660 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:50.085617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" event={"ID":"cb912f3a-ca31-4e5a-ae84-905f9198d5bc","Type":"ContainerStarted","Data":"2977e77fed234ed7ef115fc2642c601579a53ed06eeccb69d63c97b6fe434b3b"} Jul 15 21:00:50.087620 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:50.087594 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4ltr5" event={"ID":"1270fd96-fdce-4907-858d-8047350e1ee5","Type":"ContainerStarted","Data":"fd1c4153620e66bd03ec0490d76234ac07764ad3f9f5fc8898bfada2092058b4"} Jul 15 21:00:50.087725 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:50.087648 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4ltr5" event={"ID":"1270fd96-fdce-4907-858d-8047350e1ee5","Type":"ContainerStarted","Data":"b5c082c8986f9e08a8e31ef2710d7ba01cc53815cbf9240bf96414146fa486ea"} Jul 15 21:00:50.088698 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:50.088678 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" event={"ID":"3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9","Type":"ContainerStarted","Data":"064afe380613fac49b5d124f8cdbec66156777fb7a7a8a5b1d23b740d2d3c938"} Jul 15 21:00:50.120981 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:50.120921 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4ltr5" podStartSLOduration=4.371829207 podStartE2EDuration="6.120903759s" totalImagesPullingTime="1.637243141s" totalInitContainerRuntime="111.831411ms" isStatefulPod=true podCreationTimestamp="2026-07-15 21:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:50.117237198 +0000 UTC m=+103.207470379" watchObservedRunningTime="2026-07-15 21:00:50.120903759 +0000 UTC m=+103.211136927" Jul 15 21:00:51.096407 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:51.096368 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7","Type":"ContainerStarted","Data":"abc3d58f768e35aa06684413aa2deb1323ac9d7c0fec76fe1bd348c4d77100e9"} Jul 15 21:00:51.096866 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:51.096458 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7","Type":"ContainerStarted","Data":"6e66f60f07f5dd7e2eceb2f7988673a6d1057b4e60321acf53cd2fb0e85ca931"} Jul 15 21:00:51.096866 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:51.096504 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7","Type":"ContainerStarted","Data":"314d3e8d5d9d358742cb7998315063dd35cf974837ea59905de8b3a89eaa12e1"} Jul 15 21:00:51.096866 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:51.096542 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7","Type":"ContainerStarted","Data":"6ea2b5c2ce35d428391418528d4c1ea2799ba515c7bf5978442bc057f93cc4a2"} Jul 15 21:00:51.096866 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:51.096590 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7","Type":"ContainerStarted","Data":"b2eeca47e09f8fca8b288f330d074c784fe54cc891d69da0713e3fb09950339b"} Jul 15 21:00:51.948527 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:51.948499 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z7dvf" Jul 15 21:00:53.110376 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:53.110336 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7","Type":"ContainerStarted","Data":"6be34968baa8559b1f90e08aeb17f7c2ea45868949b9eb25d9c9ccd4ad28e4bc"} Jul 15 21:00:53.112043 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:53.112011 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" event={"ID":"cb912f3a-ca31-4e5a-ae84-905f9198d5bc","Type":"ContainerStarted","Data":"60167eb7aa9f90694ee91fbd494fb79dc25c6f25633a0ca08f0158ec5c623d90"} Jul 15 21:00:53.113517 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:53.113491 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" event={"ID":"3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9","Type":"ContainerStarted","Data":"264381203ea2613784454eaa0cce482d2205b5f4569eaf34f009256799a01dd1"} Jul 15 21:00:53.113690 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:53.113672 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:53.118609 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:53.118585 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" Jul 15 21:00:53.144783 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:53.144675 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.342766065 podStartE2EDuration="8.144657374s" totalImagesPullingTime="5.781021693s" totalInitContainerRuntime="20.869616ms" isStatefulPod=false podCreationTimestamp="2026-07-15 21:00:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:53.14201478 +0000 UTC m=+106.232247977" watchObservedRunningTime="2026-07-15 21:00:53.144657374 +0000 UTC m=+106.234890554" Jul 15 21:00:53.168275 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:53.168218 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-jlpb4" podStartSLOduration=1.838245939 podStartE2EDuration="4.16819749s" totalImagesPullingTime="2.329951551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:53.16725113 +0000 UTC m=+106.257484309" watchObservedRunningTime="2026-07-15 21:00:53.16819749 +0000 UTC m=+106.258430670" Jul 15 21:00:53.187192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:00:53.187071 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" podStartSLOduration=2.7346102979999998 podStartE2EDuration="5.187052451s" totalImagesPullingTime="2.452442153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:00:53.186863715 +0000 UTC m=+106.277096893" watchObservedRunningTime="2026-07-15 21:00:53.187052451 +0000 UTC m=+106.277285630" Jul 15 21:01:01.148211 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:01.148107 2573 generic.go:356] "Generic (PLEG): container finished" podID="3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88" containerID="cd5e64922ef45574a7de0f177f24a260c7613d8f49da7616a7fb7bd5e148bab8" exitCode=0 Jul 15 21:01:01.148211 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:01.148194 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" event={"ID":"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88","Type":"ContainerDied","Data":"cd5e64922ef45574a7de0f177f24a260c7613d8f49da7616a7fb7bd5e148bab8"} Jul 15 21:01:01.148678 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:01.148592 2573 scope.go:117] "RemoveContainer" containerID="cd5e64922ef45574a7de0f177f24a260c7613d8f49da7616a7fb7bd5e148bab8" Jul 15 21:01:01.149713 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:01.149687 2573 generic.go:356] "Generic (PLEG): container finished" podID="af1545f9-cf6b-4ba4-a00e-2c74949572c3" containerID="2340bf25c01e987ed0f990470ccbf1101b858237cb0ec288f64d047dbb1b8f3e" exitCode=0 Jul 15 21:01:01.149829 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:01.149725 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" event={"ID":"af1545f9-cf6b-4ba4-a00e-2c74949572c3","Type":"ContainerDied","Data":"2340bf25c01e987ed0f990470ccbf1101b858237cb0ec288f64d047dbb1b8f3e"} Jul 15 21:01:01.150013 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:01.149999 2573 scope.go:117] "RemoveContainer" containerID="2340bf25c01e987ed0f990470ccbf1101b858237cb0ec288f64d047dbb1b8f3e" Jul 15 21:01:02.134276 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:02.134243 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585bb7b99d-xcp26_14cd8b87-f21c-41c3-8cb8-7893d3f5fa28/router/0.log" Jul 15 21:01:02.144010 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:02.143986 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5jlnp_4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1/serve-healthcheck-canary/0.log" Jul 15 21:01:02.155035 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:02.154999 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-p2xkh" event={"ID":"af1545f9-cf6b-4ba4-a00e-2c74949572c3","Type":"ContainerStarted","Data":"4d9e927a53e917403fce3d1d54760524b6cbaf09c83e2b48a8c62814b17462b0"} Jul 15 21:01:02.156551 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:02.156531 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-xsxss" event={"ID":"3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88","Type":"ContainerStarted","Data":"4a7e4e7b55a1f6e2c30df771d931b21d5403838e5589fd5bd7ba9041490ee2f2"} Jul 15 21:01:03.003389 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.003350 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77f7469478-q45vb" podUID="f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" containerName="registry" containerID="cri-o://5a8b8f715d4c44e64d04f5beaaa33e25a992d848f1e1ddeaa38ac62d1ed95575" gracePeriod=30 Jul 15 21:01:03.163036 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.163007 2573 generic.go:356] "Generic (PLEG): container finished" podID="f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" containerID="5a8b8f715d4c44e64d04f5beaaa33e25a992d848f1e1ddeaa38ac62d1ed95575" exitCode=0 Jul 15 21:01:03.163388 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.163080 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77f7469478-q45vb" event={"ID":"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5","Type":"ContainerDied","Data":"5a8b8f715d4c44e64d04f5beaaa33e25a992d848f1e1ddeaa38ac62d1ed95575"} Jul 15 21:01:03.244451 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.244427 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 21:01:03.267893 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.267743 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-ca-trust-extracted\") pod \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " Jul 15 21:01:03.267893 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.267796 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-bound-sa-token\") pod \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " Jul 15 21:01:03.267893 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.267828 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-image-registry-private-configuration\") pod \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " Jul 15 21:01:03.267893 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.267870 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-installation-pull-secrets\") pod \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " Jul 15 21:01:03.269251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.267903 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") pod \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " Jul 15 21:01:03.269251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.267980 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-trusted-ca\") pod \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " Jul 15 21:01:03.269251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.268012 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjz4d\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-kube-api-access-tjz4d\") pod \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " Jul 15 21:01:03.269251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.268036 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-certificates\") pod \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\" (UID: \"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5\") " Jul 15 21:01:03.269251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.268651 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:01:03.269251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.268664 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:01:03.271419 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.271385 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:01:03.271419 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.271393 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:01:03.271875 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.271849 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:01:03.272751 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.272714 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-kube-api-access-tjz4d" (OuterVolumeSpecName: "kube-api-access-tjz4d") pod "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5"). InnerVolumeSpecName "kube-api-access-tjz4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:01:03.273085 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.273062 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:01:03.277649 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.277625 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" (UID: "f8a2ffcc-698c-43ae-8659-ed37b95f5fd5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:01:03.369053 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.369018 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-trusted-ca\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:01:03.369053 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.369046 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjz4d\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-kube-api-access-tjz4d\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:01:03.369053 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.369057 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-certificates\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:01:03.369300 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.369065 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-ca-trust-extracted\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:01:03.369300 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.369074 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-bound-sa-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:01:03.369300 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.369085 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-image-registry-private-configuration\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:01:03.369300 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.369094 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-installation-pull-secrets\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:01:03.369300 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:03.369102 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5-registry-tls\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:01:04.168592 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:04.168562 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77f7469478-q45vb" Jul 15 21:01:04.168999 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:04.168566 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77f7469478-q45vb" event={"ID":"f8a2ffcc-698c-43ae-8659-ed37b95f5fd5","Type":"ContainerDied","Data":"81748294cb5a874c38320a6c36ae113ede2598fabbbb6ee9785bfff2ddee38a3"} Jul 15 21:01:04.168999 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:04.168715 2573 scope.go:117] "RemoveContainer" containerID="5a8b8f715d4c44e64d04f5beaaa33e25a992d848f1e1ddeaa38ac62d1ed95575" Jul 15 21:01:04.189546 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:04.189520 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77f7469478-q45vb"] Jul 15 21:01:04.194067 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:04.194047 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77f7469478-q45vb"] Jul 15 21:01:05.446007 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:05.445970 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" path="/var/lib/kubelet/pods/f8a2ffcc-698c-43ae-8659-ed37b95f5fd5/volumes" Jul 15 21:01:07.185443 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:07.185413 2573 generic.go:356] "Generic (PLEG): container finished" podID="a35d610b-d08f-47d4-99c9-9f6aa55fa6ff" containerID="b1e9fc455e1479b2b7246bb354599433adde6833b5a71faca9cba73fbe4626e4" exitCode=0 Jul 15 21:01:07.185797 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:07.185463 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" event={"ID":"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff","Type":"ContainerDied","Data":"b1e9fc455e1479b2b7246bb354599433adde6833b5a71faca9cba73fbe4626e4"} Jul 15 21:01:07.185797 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:07.185757 2573 scope.go:117] "RemoveContainer" containerID="b1e9fc455e1479b2b7246bb354599433adde6833b5a71faca9cba73fbe4626e4" Jul 15 21:01:08.190585 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:08.190550 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-rvmvc" event={"ID":"a35d610b-d08f-47d4-99c9-9f6aa55fa6ff","Type":"ContainerStarted","Data":"70d39be7f21cdbc8bf7d99e550346e6d7289b88f93a675b688a2e08c6df328c0"} Jul 15 21:01:09.255771 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:09.255735 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:01:09.256118 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:09.255835 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:01:29.260685 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:29.260655 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:01:29.264700 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:01:29.264675 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b6d894486-d76t9" Jul 15 21:02:55.978599 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.978567 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v"] Jul 15 21:02:55.979040 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.978871 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" containerName="registry" Jul 15 21:02:55.979040 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.978881 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" containerName="registry" Jul 15 21:02:55.979040 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.978939 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8a2ffcc-698c-43ae-8659-ed37b95f5fd5" containerName="registry" Jul 15 21:02:55.982014 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.981997 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:55.985348 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.985323 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 21:02:55.985660 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.985644 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 21:02:55.986582 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.986555 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rsxdv\"" Jul 15 21:02:55.993568 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.993549 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5svr\" (UniqueName: \"kubernetes.io/projected/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-kube-api-access-q5svr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:55.993647 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.993585 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:55.993687 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.993661 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:55.996386 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:55.996362 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v"] Jul 15 21:02:56.094638 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:56.094603 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5svr\" (UniqueName: \"kubernetes.io/projected/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-kube-api-access-q5svr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:56.094780 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:56.094650 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:56.094780 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:56.094697 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:56.095081 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:56.095062 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:56.095117 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:56.095093 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:56.105119 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:56.105089 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5svr\" (UniqueName: \"kubernetes.io/projected/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-kube-api-access-q5svr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:56.291137 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:56.291108 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:02:56.418490 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:56.418462 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v"] Jul 15 21:02:56.421082 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:02:56.421051 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1e4191_df86_4a0b_9dab_bfe2e49ef488.slice/crio-aae868a563d96e814d4af89fc5b539eee558df5a733a6f87180582b511e12f99 WatchSource:0}: Error finding container aae868a563d96e814d4af89fc5b539eee558df5a733a6f87180582b511e12f99: Status 404 returned error can't find the container with id aae868a563d96e814d4af89fc5b539eee558df5a733a6f87180582b511e12f99 Jul 15 21:02:56.600544 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:02:56.600465 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" event={"ID":"0c1e4191-df86-4a0b-9dab-bfe2e49ef488","Type":"ContainerStarted","Data":"aae868a563d96e814d4af89fc5b539eee558df5a733a6f87180582b511e12f99"} Jul 15 21:03:02.627925 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:02.627889 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerID="c7f82435f21169bff38e8ec90e405e003527356833f460b72832e12e23db2c63" exitCode=0 Jul 15 21:03:02.628321 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:02.627964 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" event={"ID":"0c1e4191-df86-4a0b-9dab-bfe2e49ef488","Type":"ContainerDied","Data":"c7f82435f21169bff38e8ec90e405e003527356833f460b72832e12e23db2c63"} Jul 15 21:03:05.641595 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:05.641556 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerID="d8dbf6336ed90ef75a756e5ff34f90c50e3e8b1b4b020265cb2ab5e939d511ac" exitCode=0 Jul 15 21:03:05.642042 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:05.641648 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" event={"ID":"0c1e4191-df86-4a0b-9dab-bfe2e49ef488","Type":"ContainerDied","Data":"d8dbf6336ed90ef75a756e5ff34f90c50e3e8b1b4b020265cb2ab5e939d511ac"} Jul 15 21:03:14.682483 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:14.682446 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerID="a0562d22e23269a68ea3cac8d2c145fd2e8ac88262e24b09fbf2b6831cb851b2" exitCode=0 Jul 15 21:03:14.682834 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:14.682513 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" event={"ID":"0c1e4191-df86-4a0b-9dab-bfe2e49ef488","Type":"ContainerDied","Data":"a0562d22e23269a68ea3cac8d2c145fd2e8ac88262e24b09fbf2b6831cb851b2"} Jul 15 21:03:15.813730 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.813706 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:03:15.855364 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.855338 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5svr\" (UniqueName: \"kubernetes.io/projected/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-kube-api-access-q5svr\") pod \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " Jul 15 21:03:15.855464 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.855379 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-util\") pod \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " Jul 15 21:03:15.855464 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.855453 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-bundle\") pod \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\" (UID: \"0c1e4191-df86-4a0b-9dab-bfe2e49ef488\") " Jul 15 21:03:15.856064 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.856038 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-bundle" (OuterVolumeSpecName: "bundle") pod "0c1e4191-df86-4a0b-9dab-bfe2e49ef488" (UID: "0c1e4191-df86-4a0b-9dab-bfe2e49ef488"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:03:15.857409 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.857388 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-kube-api-access-q5svr" (OuterVolumeSpecName: "kube-api-access-q5svr") pod "0c1e4191-df86-4a0b-9dab-bfe2e49ef488" (UID: "0c1e4191-df86-4a0b-9dab-bfe2e49ef488"). InnerVolumeSpecName "kube-api-access-q5svr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:03:15.861347 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.861321 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-util" (OuterVolumeSpecName: "util") pod "0c1e4191-df86-4a0b-9dab-bfe2e49ef488" (UID: "0c1e4191-df86-4a0b-9dab-bfe2e49ef488"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:03:15.956928 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.956835 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-bundle\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:03:15.956928 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.956876 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5svr\" (UniqueName: \"kubernetes.io/projected/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-kube-api-access-q5svr\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:03:15.956928 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:15.956890 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0c1e4191-df86-4a0b-9dab-bfe2e49ef488-util\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:03:16.691140 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:16.691111 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" Jul 15 21:03:16.691354 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:16.691103 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402pr6v" event={"ID":"0c1e4191-df86-4a0b-9dab-bfe2e49ef488","Type":"ContainerDied","Data":"aae868a563d96e814d4af89fc5b539eee558df5a733a6f87180582b511e12f99"} Jul 15 21:03:16.691354 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:16.691228 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aae868a563d96e814d4af89fc5b539eee558df5a733a6f87180582b511e12f99" Jul 15 21:03:24.045713 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.045679 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6"] Jul 15 21:03:24.046077 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.045995 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerName="util" Jul 15 21:03:24.046077 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.046007 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerName="util" Jul 15 21:03:24.046077 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.046022 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerName="extract" Jul 15 21:03:24.046077 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.046027 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerName="extract" Jul 15 21:03:24.046077 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.046043 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerName="pull" Jul 15 21:03:24.046077 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.046049 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerName="pull" Jul 15 21:03:24.046275 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.046095 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c1e4191-df86-4a0b-9dab-bfe2e49ef488" containerName="extract" Jul 15 21:03:24.048677 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.048661 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" Jul 15 21:03:24.052109 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.052093 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zkvf4\"" Jul 15 21:03:24.052473 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.052456 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 21:03:24.052581 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.052521 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 21:03:24.065484 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.065462 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6"] Jul 15 21:03:24.123497 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.123467 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4728738-c4b4-4196-acd2-0a0dc4985ad9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9fnc6\" (UID: \"c4728738-c4b4-4196-acd2-0a0dc4985ad9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" Jul 15 21:03:24.123627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.123515 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk9bq\" (UniqueName: \"kubernetes.io/projected/c4728738-c4b4-4196-acd2-0a0dc4985ad9-kube-api-access-hk9bq\") pod \"cert-manager-operator-controller-manager-55c444bf86-9fnc6\" (UID: \"c4728738-c4b4-4196-acd2-0a0dc4985ad9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" Jul 15 21:03:24.224456 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.224428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4728738-c4b4-4196-acd2-0a0dc4985ad9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9fnc6\" (UID: \"c4728738-c4b4-4196-acd2-0a0dc4985ad9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" Jul 15 21:03:24.224575 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.224476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk9bq\" (UniqueName: \"kubernetes.io/projected/c4728738-c4b4-4196-acd2-0a0dc4985ad9-kube-api-access-hk9bq\") pod \"cert-manager-operator-controller-manager-55c444bf86-9fnc6\" (UID: \"c4728738-c4b4-4196-acd2-0a0dc4985ad9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" Jul 15 21:03:24.224854 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.224837 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4728738-c4b4-4196-acd2-0a0dc4985ad9-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9fnc6\" (UID: \"c4728738-c4b4-4196-acd2-0a0dc4985ad9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" Jul 15 21:03:24.249627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.249597 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk9bq\" (UniqueName: \"kubernetes.io/projected/c4728738-c4b4-4196-acd2-0a0dc4985ad9-kube-api-access-hk9bq\") pod \"cert-manager-operator-controller-manager-55c444bf86-9fnc6\" (UID: \"c4728738-c4b4-4196-acd2-0a0dc4985ad9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" Jul 15 21:03:24.357905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.357803 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" Jul 15 21:03:24.496013 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.495986 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6"] Jul 15 21:03:24.499017 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:03:24.498989 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4728738_c4b4_4196_acd2_0a0dc4985ad9.slice/crio-d44b0dd7e6293646a14fc210bdaf2ee2bd91ed36202244053e96a7efc29a0309 WatchSource:0}: Error finding container d44b0dd7e6293646a14fc210bdaf2ee2bd91ed36202244053e96a7efc29a0309: Status 404 returned error can't find the container with id d44b0dd7e6293646a14fc210bdaf2ee2bd91ed36202244053e96a7efc29a0309 Jul 15 21:03:24.725584 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:24.725498 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" event={"ID":"c4728738-c4b4-4196-acd2-0a0dc4985ad9","Type":"ContainerStarted","Data":"d44b0dd7e6293646a14fc210bdaf2ee2bd91ed36202244053e96a7efc29a0309"} Jul 15 21:03:28.743907 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:28.743869 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" event={"ID":"c4728738-c4b4-4196-acd2-0a0dc4985ad9","Type":"ContainerStarted","Data":"1b774d91ce7769afee8fc8d24d0920ef3f26c0f5afb3eb665e695a28cf3ba312"} Jul 15 21:03:28.774532 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:28.774483 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9fnc6" podStartSLOduration=1.575904549 podStartE2EDuration="4.774468177s" totalImagesPullingTime="3.198563628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:28.772258171 +0000 UTC m=+261.862491351" watchObservedRunningTime="2026-07-15 21:03:28.774468177 +0000 UTC m=+261.864701354" Jul 15 21:03:29.418671 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.418590 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8"] Jul 15 21:03:29.421285 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.421267 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:29.427267 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.427234 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 21:03:29.427422 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.427306 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rsxdv\"" Jul 15 21:03:29.428386 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.428367 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 21:03:29.455125 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.455098 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8"] Jul 15 21:03:29.570679 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.570649 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:29.570853 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.570685 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:29.570853 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.570725 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plp9f\" (UniqueName: \"kubernetes.io/projected/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-kube-api-access-plp9f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:29.671847 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.671762 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:29.671847 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.671810 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:29.672013 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.671867 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plp9f\" (UniqueName: \"kubernetes.io/projected/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-kube-api-access-plp9f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:29.672238 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.672216 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:29.672280 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.672223 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:29.751950 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:29.751917 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plp9f\" (UniqueName: \"kubernetes.io/projected/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-kube-api-access-plp9f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:30.030648 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.030619 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:30.186024 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.185989 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8"] Jul 15 21:03:30.188855 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:03:30.188829 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd908231f_37f3_42f6_a5c2_3a7fae2dfb8f.slice/crio-d76a9f02194d61dca6b02af119295aa8c63e8533caa42dc2bddc2d0ffe9f3219 WatchSource:0}: Error finding container d76a9f02194d61dca6b02af119295aa8c63e8533caa42dc2bddc2d0ffe9f3219: Status 404 returned error can't find the container with id d76a9f02194d61dca6b02af119295aa8c63e8533caa42dc2bddc2d0ffe9f3219 Jul 15 21:03:30.435862 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.435823 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l"] Jul 15 21:03:30.438282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.438266 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" Jul 15 21:03:30.441180 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.441148 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 21:03:30.441283 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.441200 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 21:03:30.441283 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.441249 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-b798n\"" Jul 15 21:03:30.454332 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.454311 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l"] Jul 15 21:03:30.579573 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.579482 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ce009f3-1518-47d7-9687-daf7d7b67084-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b8d6l\" (UID: \"2ce009f3-1518-47d7-9687-daf7d7b67084\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" Jul 15 21:03:30.579706 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.579614 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf875\" (UniqueName: \"kubernetes.io/projected/2ce009f3-1518-47d7-9687-daf7d7b67084-kube-api-access-mf875\") pod \"cert-manager-cainjector-559cdd8c87-b8d6l\" (UID: \"2ce009f3-1518-47d7-9687-daf7d7b67084\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" Jul 15 21:03:30.680399 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.680362 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf875\" (UniqueName: \"kubernetes.io/projected/2ce009f3-1518-47d7-9687-daf7d7b67084-kube-api-access-mf875\") pod \"cert-manager-cainjector-559cdd8c87-b8d6l\" (UID: \"2ce009f3-1518-47d7-9687-daf7d7b67084\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" Jul 15 21:03:30.680560 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.680405 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ce009f3-1518-47d7-9687-daf7d7b67084-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b8d6l\" (UID: \"2ce009f3-1518-47d7-9687-daf7d7b67084\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" Jul 15 21:03:30.690938 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.690908 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ce009f3-1518-47d7-9687-daf7d7b67084-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-b8d6l\" (UID: \"2ce009f3-1518-47d7-9687-daf7d7b67084\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" Jul 15 21:03:30.691044 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.690998 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf875\" (UniqueName: \"kubernetes.io/projected/2ce009f3-1518-47d7-9687-daf7d7b67084-kube-api-access-mf875\") pod \"cert-manager-cainjector-559cdd8c87-b8d6l\" (UID: \"2ce009f3-1518-47d7-9687-daf7d7b67084\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" Jul 15 21:03:30.754548 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.754505 2573 generic.go:356] "Generic (PLEG): container finished" podID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerID="01f159f948c4b9da2f6c88c56b4ac3fd3a4b7c73331e3aa9f31351dcf0badefa" exitCode=0 Jul 15 21:03:30.754895 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.754587 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" event={"ID":"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f","Type":"ContainerDied","Data":"01f159f948c4b9da2f6c88c56b4ac3fd3a4b7c73331e3aa9f31351dcf0badefa"} Jul 15 21:03:30.754895 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.754644 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" event={"ID":"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f","Type":"ContainerStarted","Data":"d76a9f02194d61dca6b02af119295aa8c63e8533caa42dc2bddc2d0ffe9f3219"} Jul 15 21:03:30.759706 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.759682 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" Jul 15 21:03:30.894856 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:30.894832 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l"] Jul 15 21:03:30.897322 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:03:30.897284 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ce009f3_1518_47d7_9687_daf7d7b67084.slice/crio-ea255892e4133486d1e4a5a4ff9744a4d2aad9c640b9a15b91706f5cd9688ba5 WatchSource:0}: Error finding container ea255892e4133486d1e4a5a4ff9744a4d2aad9c640b9a15b91706f5cd9688ba5: Status 404 returned error can't find the container with id ea255892e4133486d1e4a5a4ff9744a4d2aad9c640b9a15b91706f5cd9688ba5 Jul 15 21:03:31.761828 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:31.761787 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" event={"ID":"2ce009f3-1518-47d7-9687-daf7d7b67084","Type":"ContainerStarted","Data":"ea255892e4133486d1e4a5a4ff9744a4d2aad9c640b9a15b91706f5cd9688ba5"} Jul 15 21:03:34.778798 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:34.778764 2573 generic.go:356] "Generic (PLEG): container finished" podID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerID="e87dd7f1ddcb75b3fa652f06afd9ad99860886c037f6e74b4c051e8360b9cd85" exitCode=0 Jul 15 21:03:34.779243 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:34.778839 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" event={"ID":"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f","Type":"ContainerDied","Data":"e87dd7f1ddcb75b3fa652f06afd9ad99860886c037f6e74b4c051e8360b9cd85"} Jul 15 21:03:34.780398 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:34.780369 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" event={"ID":"2ce009f3-1518-47d7-9687-daf7d7b67084","Type":"ContainerStarted","Data":"696ffbc4c1bacaf909d4102918fc270d4a8100006af8c5ed522292db48c9360f"} Jul 15 21:03:34.822360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:34.822306 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-b8d6l" podStartSLOduration=1.688507124 podStartE2EDuration="4.822283477s" totalImagesPullingTime="3.133776353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:34.820009602 +0000 UTC m=+267.910242780" watchObservedRunningTime="2026-07-15 21:03:34.822283477 +0000 UTC m=+267.912516656" Jul 15 21:03:35.787081 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:35.787044 2573 generic.go:356] "Generic (PLEG): container finished" podID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerID="f2ddd043eeb08487f166a2fa9cc040b67d840ef7577b3405df53fdffee54a747" exitCode=0 Jul 15 21:03:35.787508 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:35.787108 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" event={"ID":"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f","Type":"ContainerDied","Data":"f2ddd043eeb08487f166a2fa9cc040b67d840ef7577b3405df53fdffee54a747"} Jul 15 21:03:36.921521 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:36.921492 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:37.039387 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.039346 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-util\") pod \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " Jul 15 21:03:37.039563 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.039448 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plp9f\" (UniqueName: \"kubernetes.io/projected/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-kube-api-access-plp9f\") pod \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " Jul 15 21:03:37.039563 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.039480 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-bundle\") pod \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\" (UID: \"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f\") " Jul 15 21:03:37.039952 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.039922 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-bundle" (OuterVolumeSpecName: "bundle") pod "d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" (UID: "d908231f-37f3-42f6-a5c2-3a7fae2dfb8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:03:37.041621 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.041564 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-kube-api-access-plp9f" (OuterVolumeSpecName: "kube-api-access-plp9f") pod "d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" (UID: "d908231f-37f3-42f6-a5c2-3a7fae2dfb8f"). InnerVolumeSpecName "kube-api-access-plp9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:03:37.044687 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.044655 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-util" (OuterVolumeSpecName: "util") pod "d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" (UID: "d908231f-37f3-42f6-a5c2-3a7fae2dfb8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:03:37.140086 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.140038 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plp9f\" (UniqueName: \"kubernetes.io/projected/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-kube-api-access-plp9f\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:03:37.140086 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.140081 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-bundle\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:03:37.140086 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.140093 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d908231f-37f3-42f6-a5c2-3a7fae2dfb8f-util\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:03:37.797188 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.797147 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" event={"ID":"d908231f-37f3-42f6-a5c2-3a7fae2dfb8f","Type":"ContainerDied","Data":"d76a9f02194d61dca6b02af119295aa8c63e8533caa42dc2bddc2d0ffe9f3219"} Jul 15 21:03:37.797344 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.797224 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d76a9f02194d61dca6b02af119295aa8c63e8533caa42dc2bddc2d0ffe9f3219" Jul 15 21:03:37.797344 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:37.797165 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsmdg8" Jul 15 21:03:44.074200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.071300 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4"] Jul 15 21:03:44.074200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.072069 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerName="util" Jul 15 21:03:44.074200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.072089 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerName="util" Jul 15 21:03:44.074200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.072120 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerName="pull" Jul 15 21:03:44.074200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.072128 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerName="pull" Jul 15 21:03:44.074200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.072145 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerName="extract" Jul 15 21:03:44.074200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.072154 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerName="extract" Jul 15 21:03:44.074200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.072324 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d908231f-37f3-42f6-a5c2-3a7fae2dfb8f" containerName="extract" Jul 15 21:03:44.077356 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.077331 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" Jul 15 21:03:44.080341 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.080321 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 21:03:44.080447 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.080324 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 21:03:44.081196 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.081180 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lxvz7\"" Jul 15 21:03:44.083788 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.083766 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4"] Jul 15 21:03:44.203390 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.203353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52150eab-8243-47dd-8b9d-283896f16a8e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g7hn4\" (UID: \"52150eab-8243-47dd-8b9d-283896f16a8e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" Jul 15 21:03:44.203549 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.203396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lklzl\" (UniqueName: \"kubernetes.io/projected/52150eab-8243-47dd-8b9d-283896f16a8e-kube-api-access-lklzl\") pod \"openshift-lws-operator-fd8ccff4c-g7hn4\" (UID: \"52150eab-8243-47dd-8b9d-283896f16a8e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" Jul 15 21:03:44.304328 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.304291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52150eab-8243-47dd-8b9d-283896f16a8e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g7hn4\" (UID: \"52150eab-8243-47dd-8b9d-283896f16a8e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" Jul 15 21:03:44.304519 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.304347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lklzl\" (UniqueName: \"kubernetes.io/projected/52150eab-8243-47dd-8b9d-283896f16a8e-kube-api-access-lklzl\") pod \"openshift-lws-operator-fd8ccff4c-g7hn4\" (UID: \"52150eab-8243-47dd-8b9d-283896f16a8e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" Jul 15 21:03:44.304728 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.304707 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52150eab-8243-47dd-8b9d-283896f16a8e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g7hn4\" (UID: \"52150eab-8243-47dd-8b9d-283896f16a8e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" Jul 15 21:03:44.326325 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.326255 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lklzl\" (UniqueName: \"kubernetes.io/projected/52150eab-8243-47dd-8b9d-283896f16a8e-kube-api-access-lklzl\") pod \"openshift-lws-operator-fd8ccff4c-g7hn4\" (UID: \"52150eab-8243-47dd-8b9d-283896f16a8e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" Jul 15 21:03:44.387315 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.387280 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" Jul 15 21:03:44.516260 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.516230 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4"] Jul 15 21:03:44.518928 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:03:44.518902 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52150eab_8243_47dd_8b9d_283896f16a8e.slice/crio-2674a89be8e217fd5b64b398bf849d17c00dee21a5c7843cb8f17caa13811da9 WatchSource:0}: Error finding container 2674a89be8e217fd5b64b398bf849d17c00dee21a5c7843cb8f17caa13811da9: Status 404 returned error can't find the container with id 2674a89be8e217fd5b64b398bf849d17c00dee21a5c7843cb8f17caa13811da9 Jul 15 21:03:44.828949 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:44.828914 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" event={"ID":"52150eab-8243-47dd-8b9d-283896f16a8e","Type":"ContainerStarted","Data":"2674a89be8e217fd5b64b398bf849d17c00dee21a5c7843cb8f17caa13811da9"} Jul 15 21:03:46.841876 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:46.841836 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" event={"ID":"52150eab-8243-47dd-8b9d-283896f16a8e","Type":"ContainerStarted","Data":"d932e0b0ff9e99d508d9b6ee8332c8a1afc0ac85e57e2651166a1fdf2d1ec235"} Jul 15 21:03:46.862143 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:46.862085 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g7hn4" podStartSLOduration=0.991090104 podStartE2EDuration="2.862063198s" totalImagesPullingTime="1.870973094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:46.859199121 +0000 UTC m=+279.949432298" watchObservedRunningTime="2026-07-15 21:03:46.862063198 +0000 UTC m=+279.952296379" Jul 15 21:03:49.157923 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.157887 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sj2fn"] Jul 15 21:03:49.179900 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.179874 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sj2fn"] Jul 15 21:03:49.180026 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.179961 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sj2fn" Jul 15 21:03:49.183555 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.183533 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sqxk4\"" Jul 15 21:03:49.244452 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.244426 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/433abfa4-fd40-4c40-b728-ea8d1cfe7217-bound-sa-token\") pod \"cert-manager-55b4d954d5-sj2fn\" (UID: \"433abfa4-fd40-4c40-b728-ea8d1cfe7217\") " pod="cert-manager/cert-manager-55b4d954d5-sj2fn" Jul 15 21:03:49.244594 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.244474 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gffwt\" (UniqueName: \"kubernetes.io/projected/433abfa4-fd40-4c40-b728-ea8d1cfe7217-kube-api-access-gffwt\") pod \"cert-manager-55b4d954d5-sj2fn\" (UID: \"433abfa4-fd40-4c40-b728-ea8d1cfe7217\") " pod="cert-manager/cert-manager-55b4d954d5-sj2fn" Jul 15 21:03:49.344936 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.344903 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/433abfa4-fd40-4c40-b728-ea8d1cfe7217-bound-sa-token\") pod \"cert-manager-55b4d954d5-sj2fn\" (UID: \"433abfa4-fd40-4c40-b728-ea8d1cfe7217\") " pod="cert-manager/cert-manager-55b4d954d5-sj2fn" Jul 15 21:03:49.345103 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.344956 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gffwt\" (UniqueName: \"kubernetes.io/projected/433abfa4-fd40-4c40-b728-ea8d1cfe7217-kube-api-access-gffwt\") pod \"cert-manager-55b4d954d5-sj2fn\" (UID: \"433abfa4-fd40-4c40-b728-ea8d1cfe7217\") " pod="cert-manager/cert-manager-55b4d954d5-sj2fn" Jul 15 21:03:49.354405 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.354375 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/433abfa4-fd40-4c40-b728-ea8d1cfe7217-bound-sa-token\") pod \"cert-manager-55b4d954d5-sj2fn\" (UID: \"433abfa4-fd40-4c40-b728-ea8d1cfe7217\") " pod="cert-manager/cert-manager-55b4d954d5-sj2fn" Jul 15 21:03:49.354594 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.354570 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gffwt\" (UniqueName: \"kubernetes.io/projected/433abfa4-fd40-4c40-b728-ea8d1cfe7217-kube-api-access-gffwt\") pod \"cert-manager-55b4d954d5-sj2fn\" (UID: \"433abfa4-fd40-4c40-b728-ea8d1cfe7217\") " pod="cert-manager/cert-manager-55b4d954d5-sj2fn" Jul 15 21:03:49.494075 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.493995 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sj2fn" Jul 15 21:03:49.608254 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.608220 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5"] Jul 15 21:03:49.638399 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.638371 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5"] Jul 15 21:03:49.638538 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.638469 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.641521 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.641497 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 21:03:49.641652 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.641502 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rsxdv\"" Jul 15 21:03:49.641652 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.641502 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 21:03:49.647304 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.647279 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sj2fn"] Jul 15 21:03:49.647594 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.647568 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.647639 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.647620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnsnm\" (UniqueName: \"kubernetes.io/projected/0f719574-b23f-4593-bfc9-648247e1edc3-kube-api-access-vnsnm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.647692 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.647670 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.649401 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:03:49.649378 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod433abfa4_fd40_4c40_b728_ea8d1cfe7217.slice/crio-23fefef5c8cc48d44b20ca1d790ec63fc40b9990cec2d17d5602ca80229922e7 WatchSource:0}: Error finding container 23fefef5c8cc48d44b20ca1d790ec63fc40b9990cec2d17d5602ca80229922e7: Status 404 returned error can't find the container with id 23fefef5c8cc48d44b20ca1d790ec63fc40b9990cec2d17d5602ca80229922e7 Jul 15 21:03:49.749055 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.748990 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.749055 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.749033 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnsnm\" (UniqueName: \"kubernetes.io/projected/0f719574-b23f-4593-bfc9-648247e1edc3-kube-api-access-vnsnm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.749297 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.749076 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.749373 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.749356 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.749464 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.749443 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.759032 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.759007 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnsnm\" (UniqueName: \"kubernetes.io/projected/0f719574-b23f-4593-bfc9-648247e1edc3-kube-api-access-vnsnm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:49.855410 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.855373 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sj2fn" event={"ID":"433abfa4-fd40-4c40-b728-ea8d1cfe7217","Type":"ContainerStarted","Data":"0f1bd5c371ceb8cef799776c4b9cfc92c3f055b3aeb23bdc1b672d5b7d0fb59e"} Jul 15 21:03:49.855410 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.855417 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sj2fn" event={"ID":"433abfa4-fd40-4c40-b728-ea8d1cfe7217","Type":"ContainerStarted","Data":"23fefef5c8cc48d44b20ca1d790ec63fc40b9990cec2d17d5602ca80229922e7"} Jul 15 21:03:49.875676 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.875630 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-sj2fn" podStartSLOduration=0.875615628 podStartE2EDuration="875.615628ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:03:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:03:49.874481117 +0000 UTC m=+282.964714308" watchObservedRunningTime="2026-07-15 21:03:49.875615628 +0000 UTC m=+282.965848806" Jul 15 21:03:49.948769 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:49.948736 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:50.097868 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:50.097819 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5"] Jul 15 21:03:50.099972 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:03:50.099947 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f719574_b23f_4593_bfc9_648247e1edc3.slice/crio-180e4eb5162939aef1d6b36390bb11b9598c22fe7f9ddfff0cdf532d45789cae WatchSource:0}: Error finding container 180e4eb5162939aef1d6b36390bb11b9598c22fe7f9ddfff0cdf532d45789cae: Status 404 returned error can't find the container with id 180e4eb5162939aef1d6b36390bb11b9598c22fe7f9ddfff0cdf532d45789cae Jul 15 21:03:50.861296 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:50.861260 2573 generic.go:356] "Generic (PLEG): container finished" podID="0f719574-b23f-4593-bfc9-648247e1edc3" containerID="da93353a42a322ad20a197ce0af370814e847ddbd6a8358a6e6c8499141ee155" exitCode=0 Jul 15 21:03:50.861767 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:50.861328 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" event={"ID":"0f719574-b23f-4593-bfc9-648247e1edc3","Type":"ContainerDied","Data":"da93353a42a322ad20a197ce0af370814e847ddbd6a8358a6e6c8499141ee155"} Jul 15 21:03:50.861767 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:50.861390 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" event={"ID":"0f719574-b23f-4593-bfc9-648247e1edc3","Type":"ContainerStarted","Data":"180e4eb5162939aef1d6b36390bb11b9598c22fe7f9ddfff0cdf532d45789cae"} Jul 15 21:03:51.867664 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:51.867627 2573 generic.go:356] "Generic (PLEG): container finished" podID="0f719574-b23f-4593-bfc9-648247e1edc3" containerID="7c73763d6d38eed9fcb75353845a9965c5a2080cb001398aeafdfa9b742c2f90" exitCode=0 Jul 15 21:03:51.868039 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:51.867715 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" event={"ID":"0f719574-b23f-4593-bfc9-648247e1edc3","Type":"ContainerDied","Data":"7c73763d6d38eed9fcb75353845a9965c5a2080cb001398aeafdfa9b742c2f90"} Jul 15 21:03:52.880013 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:52.879975 2573 generic.go:356] "Generic (PLEG): container finished" podID="0f719574-b23f-4593-bfc9-648247e1edc3" containerID="46322af2a26e056ee5a19b4bb5201eb65470914055d6793961e7294baaf8af57" exitCode=0 Jul 15 21:03:52.880013 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:52.880015 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" event={"ID":"0f719574-b23f-4593-bfc9-648247e1edc3","Type":"ContainerDied","Data":"46322af2a26e056ee5a19b4bb5201eb65470914055d6793961e7294baaf8af57"} Jul 15 21:03:54.007960 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.007934 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:54.088239 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.088207 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-bundle\") pod \"0f719574-b23f-4593-bfc9-648247e1edc3\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " Jul 15 21:03:54.088400 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.088260 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-util\") pod \"0f719574-b23f-4593-bfc9-648247e1edc3\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " Jul 15 21:03:54.088400 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.088288 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnsnm\" (UniqueName: \"kubernetes.io/projected/0f719574-b23f-4593-bfc9-648247e1edc3-kube-api-access-vnsnm\") pod \"0f719574-b23f-4593-bfc9-648247e1edc3\" (UID: \"0f719574-b23f-4593-bfc9-648247e1edc3\") " Jul 15 21:03:54.088961 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.088936 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-bundle" (OuterVolumeSpecName: "bundle") pod "0f719574-b23f-4593-bfc9-648247e1edc3" (UID: "0f719574-b23f-4593-bfc9-648247e1edc3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:03:54.090526 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.090501 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f719574-b23f-4593-bfc9-648247e1edc3-kube-api-access-vnsnm" (OuterVolumeSpecName: "kube-api-access-vnsnm") pod "0f719574-b23f-4593-bfc9-648247e1edc3" (UID: "0f719574-b23f-4593-bfc9-648247e1edc3"). InnerVolumeSpecName "kube-api-access-vnsnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:03:54.093999 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.093963 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-util" (OuterVolumeSpecName: "util") pod "0f719574-b23f-4593-bfc9-648247e1edc3" (UID: "0f719574-b23f-4593-bfc9-648247e1edc3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:03:54.189151 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.189066 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-bundle\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:03:54.189151 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.189094 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f719574-b23f-4593-bfc9-648247e1edc3-util\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:03:54.189151 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.189104 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnsnm\" (UniqueName: \"kubernetes.io/projected/0f719574-b23f-4593-bfc9-648247e1edc3-kube-api-access-vnsnm\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:03:54.891463 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.891426 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" event={"ID":"0f719574-b23f-4593-bfc9-648247e1edc3","Type":"ContainerDied","Data":"180e4eb5162939aef1d6b36390bb11b9598c22fe7f9ddfff0cdf532d45789cae"} Jul 15 21:03:54.891463 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.891451 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmvt5" Jul 15 21:03:54.891663 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:03:54.891476 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="180e4eb5162939aef1d6b36390bb11b9598c22fe7f9ddfff0cdf532d45789cae" Jul 15 21:04:06.717190 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.717142 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb"] Jul 15 21:04:06.717561 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.717544 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f719574-b23f-4593-bfc9-648247e1edc3" containerName="pull" Jul 15 21:04:06.717604 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.717561 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f719574-b23f-4593-bfc9-648247e1edc3" containerName="pull" Jul 15 21:04:06.717604 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.717574 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f719574-b23f-4593-bfc9-648247e1edc3" containerName="util" Jul 15 21:04:06.717604 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.717580 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f719574-b23f-4593-bfc9-648247e1edc3" containerName="util" Jul 15 21:04:06.717604 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.717587 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f719574-b23f-4593-bfc9-648247e1edc3" containerName="extract" Jul 15 21:04:06.717604 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.717593 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f719574-b23f-4593-bfc9-648247e1edc3" containerName="extract" Jul 15 21:04:06.717837 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.717661 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f719574-b23f-4593-bfc9-648247e1edc3" containerName="extract" Jul 15 21:04:06.736854 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.736825 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.738781 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.738757 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb"] Jul 15 21:04:06.739903 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.739880 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 21:04:06.740232 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.740214 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 21:04:06.740650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.740630 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 21:04:06.740758 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.740638 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 21:04:06.740758 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.740674 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lktzz\"" Jul 15 21:04:06.803225 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.803166 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.803415 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.803234 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-webhook-cert\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.803415 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.803261 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-manifests\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.803415 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.803293 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsgzv\" (UniqueName: \"kubernetes.io/projected/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-kube-api-access-zsgzv\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.803550 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.803419 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-tmp\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.904257 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.904216 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-tmp\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.904445 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.904266 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.904445 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.904291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-webhook-cert\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.904445 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.904323 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-manifests\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.904445 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.904366 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsgzv\" (UniqueName: \"kubernetes.io/projected/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-kube-api-access-zsgzv\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.904733 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.904706 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-manifests\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.906578 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.906553 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-tmp\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.906868 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.906844 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.906959 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.906878 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-webhook-cert\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:06.921192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:06.921138 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsgzv\" (UniqueName: \"kubernetes.io/projected/7c6d6487-2ef6-44c1-b438-cc3c92ad3797-kube-api-access-zsgzv\") pod \"opendatahub-operator-controller-manager-74776f8bd9-xm2jb\" (UID: \"7c6d6487-2ef6-44c1-b438-cc3c92ad3797\") " pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:07.048403 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:07.048367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:07.241248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:07.241218 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb"] Jul 15 21:04:07.244295 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:04:07.244269 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6d6487_2ef6_44c1_b438_cc3c92ad3797.slice/crio-bf174a735019777afa5a00e14c32348bbef7f56b4ee159ca5fa72e2a25e75ebe WatchSource:0}: Error finding container bf174a735019777afa5a00e14c32348bbef7f56b4ee159ca5fa72e2a25e75ebe: Status 404 returned error can't find the container with id bf174a735019777afa5a00e14c32348bbef7f56b4ee159ca5fa72e2a25e75ebe Jul 15 21:04:07.306200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:07.306106 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:04:07.310243 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:07.310219 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:04:07.310387 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:07.310343 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:04:07.314582 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:07.314557 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:04:07.318088 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:07.318071 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 21:04:07.952829 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:07.952795 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" event={"ID":"7c6d6487-2ef6-44c1-b438-cc3c92ad3797","Type":"ContainerStarted","Data":"bf174a735019777afa5a00e14c32348bbef7f56b4ee159ca5fa72e2a25e75ebe"} Jul 15 21:04:10.971453 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:10.971412 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" event={"ID":"7c6d6487-2ef6-44c1-b438-cc3c92ad3797","Type":"ContainerStarted","Data":"2a6e109310b8375bfe0338215022eec70acb55fe84be6d6f080c47b5f555bef0"} Jul 15 21:04:11.977295 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:11.977163 2573 generic.go:356] "Generic (PLEG): container finished" podID="7c6d6487-2ef6-44c1-b438-cc3c92ad3797" containerID="2a6e109310b8375bfe0338215022eec70acb55fe84be6d6f080c47b5f555bef0" exitCode=0 Jul 15 21:04:11.977295 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:11.977273 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" event={"ID":"7c6d6487-2ef6-44c1-b438-cc3c92ad3797","Type":"ContainerDied","Data":"2a6e109310b8375bfe0338215022eec70acb55fe84be6d6f080c47b5f555bef0"} Jul 15 21:04:11.977879 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:11.977858 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:04:12.983543 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:12.983504 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" event={"ID":"7c6d6487-2ef6-44c1-b438-cc3c92ad3797","Type":"ContainerStarted","Data":"7894dd4141d7a52a0a5019a67fdb4d34261e3edc599760edcf16562f4c7875cc"} Jul 15 21:04:12.984158 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:12.983649 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:13.008767 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:13.008707 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" podStartSLOduration=2.471929368 podStartE2EDuration="7.008689178s" totalImagesPullingTime="3.355539782s" totalInitContainerRuntime="1.181220028s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:04:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:04:13.006734685 +0000 UTC m=+306.096967874" watchObservedRunningTime="2026-07-15 21:04:13.008689178 +0000 UTC m=+306.098922356" Jul 15 21:04:23.991211 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:23.991157 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74776f8bd9-xm2jb" Jul 15 21:04:26.715392 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.715356 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c"] Jul 15 21:04:26.725025 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.724996 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.731911 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:04:26.731872 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"lws-controller-manager-dockercfg-tp66v\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tp66v\"" type="*v1.Secret" Jul 15 21:04:26.731911 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:04:26.731878 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-cert\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret" Jul 15 21:04:26.732065 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:04:26.731878 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"lws-manager-config\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" type="*v1.ConfigMap" Jul 15 21:04:26.732065 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:04:26.731878 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"webhook-server-cert\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret" Jul 15 21:04:26.732065 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:04:26.731943 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"lws-controller-manager-649f4f97b8-qpm5c\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="099f541c-62a5-4682-b722-b519406ed95e" pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.765182 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.765143 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c"] Jul 15 21:04:26.784505 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.784471 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/099f541c-62a5-4682-b722-b519406ed95e-cert\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.784665 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.784516 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx9pc\" (UniqueName: \"kubernetes.io/projected/099f541c-62a5-4682-b722-b519406ed95e-kube-api-access-sx9pc\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.784665 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.784557 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/099f541c-62a5-4682-b722-b519406ed95e-manager-config\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.784665 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.784577 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/099f541c-62a5-4682-b722-b519406ed95e-metrics-cert\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.886014 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.885969 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx9pc\" (UniqueName: \"kubernetes.io/projected/099f541c-62a5-4682-b722-b519406ed95e-kube-api-access-sx9pc\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.886226 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.886021 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/099f541c-62a5-4682-b722-b519406ed95e-manager-config\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.886226 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.886044 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/099f541c-62a5-4682-b722-b519406ed95e-metrics-cert\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.886226 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.886085 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/099f541c-62a5-4682-b722-b519406ed95e-cert\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:26.907907 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:26.907873 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx9pc\" (UniqueName: \"kubernetes.io/projected/099f541c-62a5-4682-b722-b519406ed95e-kube-api-access-sx9pc\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:27.625196 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:27.625143 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 21:04:27.629616 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:27.629584 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/099f541c-62a5-4682-b722-b519406ed95e-cert\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:27.701873 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:27.701835 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 21:04:27.707109 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:27.707083 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/099f541c-62a5-4682-b722-b519406ed95e-manager-config\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:27.780340 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:27.780311 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 21:04:27.789211 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:27.789160 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/099f541c-62a5-4682-b722-b519406ed95e-metrics-cert\") pod \"lws-controller-manager-649f4f97b8-qpm5c\" (UID: \"099f541c-62a5-4682-b722-b519406ed95e\") " pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:28.187680 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.187643 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ccf85"] Jul 15 21:04:28.190210 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.190164 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:28.193023 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.193003 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 21:04:28.193958 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.193940 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-8lpjc\"" Jul 15 21:04:28.199067 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.199045 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhpck\" (UniqueName: \"kubernetes.io/projected/85601b0c-aa09-4e72-a515-784c4d9661ba-kube-api-access-xhpck\") pod \"model-serving-api-54d89bd5fc-ccf85\" (UID: \"85601b0c-aa09-4e72-a515-784c4d9661ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:28.199155 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.199078 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85601b0c-aa09-4e72-a515-784c4d9661ba-tls-certs\") pod \"model-serving-api-54d89bd5fc-ccf85\" (UID: \"85601b0c-aa09-4e72-a515-784c4d9661ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:28.202952 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.202929 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ccf85"] Jul 15 21:04:28.299817 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.299785 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhpck\" (UniqueName: \"kubernetes.io/projected/85601b0c-aa09-4e72-a515-784c4d9661ba-kube-api-access-xhpck\") pod \"model-serving-api-54d89bd5fc-ccf85\" (UID: \"85601b0c-aa09-4e72-a515-784c4d9661ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:28.299817 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.299818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85601b0c-aa09-4e72-a515-784c4d9661ba-tls-certs\") pod \"model-serving-api-54d89bd5fc-ccf85\" (UID: \"85601b0c-aa09-4e72-a515-784c4d9661ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:28.300016 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:04:28.299942 2573 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 15 21:04:28.300016 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:04:28.299999 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85601b0c-aa09-4e72-a515-784c4d9661ba-tls-certs podName:85601b0c-aa09-4e72-a515-784c4d9661ba nodeName:}" failed. No retries permitted until 2026-07-15 21:04:28.799983827 +0000 UTC m=+321.890216984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/85601b0c-aa09-4e72-a515-784c4d9661ba-tls-certs") pod "model-serving-api-54d89bd5fc-ccf85" (UID: "85601b0c-aa09-4e72-a515-784c4d9661ba") : secret "model-serving-api-tls" not found Jul 15 21:04:28.309997 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.309970 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhpck\" (UniqueName: \"kubernetes.io/projected/85601b0c-aa09-4e72-a515-784c4d9661ba-kube-api-access-xhpck\") pod \"model-serving-api-54d89bd5fc-ccf85\" (UID: \"85601b0c-aa09-4e72-a515-784c4d9661ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:28.331339 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.331314 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tp66v\"" Jul 15 21:04:28.334674 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.334657 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:28.471586 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.471512 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c"] Jul 15 21:04:28.474777 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:04:28.474752 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099f541c_62a5_4682_b722_b519406ed95e.slice/crio-aaa0784b038314b7cea1040d801a9948ce6b42e4ba906764b8ab0741fa6d92d4 WatchSource:0}: Error finding container aaa0784b038314b7cea1040d801a9948ce6b42e4ba906764b8ab0741fa6d92d4: Status 404 returned error can't find the container with id aaa0784b038314b7cea1040d801a9948ce6b42e4ba906764b8ab0741fa6d92d4 Jul 15 21:04:28.805520 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.805480 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85601b0c-aa09-4e72-a515-784c4d9661ba-tls-certs\") pod \"model-serving-api-54d89bd5fc-ccf85\" (UID: \"85601b0c-aa09-4e72-a515-784c4d9661ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:28.807885 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:28.807866 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85601b0c-aa09-4e72-a515-784c4d9661ba-tls-certs\") pod \"model-serving-api-54d89bd5fc-ccf85\" (UID: \"85601b0c-aa09-4e72-a515-784c4d9661ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:29.065912 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:29.065819 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" event={"ID":"099f541c-62a5-4682-b722-b519406ed95e","Type":"ContainerStarted","Data":"aaa0784b038314b7cea1040d801a9948ce6b42e4ba906764b8ab0741fa6d92d4"} Jul 15 21:04:29.101292 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:29.101228 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:29.283943 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:29.283851 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-ccf85"] Jul 15 21:04:29.286580 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:04:29.286553 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85601b0c_aa09_4e72_a515_784c4d9661ba.slice/crio-729e80f2a6d8dad1d62bba7bdb5a09fea6570a6af4399fdf6da66c3dda4ee58b WatchSource:0}: Error finding container 729e80f2a6d8dad1d62bba7bdb5a09fea6570a6af4399fdf6da66c3dda4ee58b: Status 404 returned error can't find the container with id 729e80f2a6d8dad1d62bba7bdb5a09fea6570a6af4399fdf6da66c3dda4ee58b Jul 15 21:04:30.071784 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:30.071753 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" event={"ID":"85601b0c-aa09-4e72-a515-784c4d9661ba","Type":"ContainerStarted","Data":"729e80f2a6d8dad1d62bba7bdb5a09fea6570a6af4399fdf6da66c3dda4ee58b"} Jul 15 21:04:30.073445 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:30.073419 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" event={"ID":"099f541c-62a5-4682-b722-b519406ed95e","Type":"ContainerStarted","Data":"0b4598c705dee20b6209ad8ee45b190674aa01de9f5caae989565c64e3ae1ddc"} Jul 15 21:04:30.073579 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:30.073557 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:30.105775 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:30.105722 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" podStartSLOduration=2.586360635 podStartE2EDuration="4.105703112s" totalImagesPullingTime="1.519342477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:04:30.103837631 +0000 UTC m=+323.194070809" watchObservedRunningTime="2026-07-15 21:04:30.105703112 +0000 UTC m=+323.195936289" Jul 15 21:04:32.085499 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:32.085455 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" event={"ID":"85601b0c-aa09-4e72-a515-784c4d9661ba","Type":"ContainerStarted","Data":"d73db4c016c1bb85e12f8d40cd899f2955e5a9a5f3dba3450b62fdb7dd2077de"} Jul 15 21:04:32.085906 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:32.085654 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:32.107918 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:32.107874 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" podStartSLOduration=2.189783208 podStartE2EDuration="4.107860056s" totalImagesPullingTime="1.918076848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:04:32.107166472 +0000 UTC m=+325.197399650" watchObservedRunningTime="2026-07-15 21:04:32.107860056 +0000 UTC m=+325.198093285" Jul 15 21:04:34.322151 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.322110 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6"] Jul 15 21:04:34.324721 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.324698 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.330003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.329978 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-d2bjs\"" Jul 15 21:04:34.330358 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.330331 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 21:04:34.349927 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.349901 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6"] Jul 15 21:04:34.353264 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.353236 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.353360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.353281 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.353360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.353314 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.353441 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.353378 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc055e40-aa14-4337-8136-c1f7e5d51589-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.353441 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.353412 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.353537 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.353492 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.353537 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.353516 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql7mz\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-kube-api-access-ql7mz\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.353622 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.353589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.353706 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.353667 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455101 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455124 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455153 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455197 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455420 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455319 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455420 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455367 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc055e40-aa14-4337-8136-c1f7e5d51589-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455420 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455400 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455565 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455500 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455565 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455532 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455565 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455555 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455773 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455565 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql7mz\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-kube-api-access-ql7mz\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455773 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455601 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.455773 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.455763 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.456094 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.456065 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc055e40-aa14-4337-8136-c1f7e5d51589-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.457876 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.457855 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.458433 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.458413 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.459276 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.459257 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s"] Jul 15 21:04:34.461687 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.461672 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.466137 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.466114 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.477605 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.477585 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql7mz\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-kube-api-access-ql7mz\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.481572 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.481550 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s"] Jul 15 21:04:34.556794 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.556758 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36b9ccbb-9fa2-4561-b234-49b301aeee60-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.556980 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.556866 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.556980 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.556902 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.556980 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.556920 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjgz4\" (UniqueName: \"kubernetes.io/projected/36b9ccbb-9fa2-4561-b234-49b301aeee60-kube-api-access-vjgz4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.556980 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.556949 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.556980 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.556970 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.557192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.556990 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.557192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.557006 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.557192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.557050 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.635375 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.635294 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:34.657680 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.657651 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.657839 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.657686 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.657839 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.657704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjgz4\" (UniqueName: \"kubernetes.io/projected/36b9ccbb-9fa2-4561-b234-49b301aeee60-kube-api-access-vjgz4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.657839 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.657731 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.657999 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.657847 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.657999 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.657882 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.657999 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.657898 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.657999 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.657962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.658250 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.658004 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36b9ccbb-9fa2-4561-b234-49b301aeee60-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.658250 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.658150 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.658438 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.658417 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.658483 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.658467 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.658549 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.658530 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.658761 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.658725 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36b9ccbb-9fa2-4561-b234-49b301aeee60-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.659965 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.659945 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.660465 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.660449 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.668185 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.668148 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36b9ccbb-9fa2-4561-b234-49b301aeee60-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.668626 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.668609 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjgz4\" (UniqueName: \"kubernetes.io/projected/36b9ccbb-9fa2-4561-b234-49b301aeee60-kube-api-access-vjgz4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46vn9s\" (UID: \"36b9ccbb-9fa2-4561-b234-49b301aeee60\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.787956 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.787928 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6"] Jul 15 21:04:34.789152 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:04:34.789126 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc055e40_aa14_4337_8136_c1f7e5d51589.slice/crio-483c068db416aca756e02fd5a94e9ccb988da8a1f9abaa0bffef49a3318d40d0 WatchSource:0}: Error finding container 483c068db416aca756e02fd5a94e9ccb988da8a1f9abaa0bffef49a3318d40d0: Status 404 returned error can't find the container with id 483c068db416aca756e02fd5a94e9ccb988da8a1f9abaa0bffef49a3318d40d0 Jul 15 21:04:34.790735 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.790707 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:34.927258 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:34.927226 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s"] Jul 15 21:04:34.929027 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:04:34.928992 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b9ccbb_9fa2_4561_b234_49b301aeee60.slice/crio-6cdbe9f9d4ed7250fdb7850d511aab83623d033acc575cb854f1346d6ccca1c6 WatchSource:0}: Error finding container 6cdbe9f9d4ed7250fdb7850d511aab83623d033acc575cb854f1346d6ccca1c6: Status 404 returned error can't find the container with id 6cdbe9f9d4ed7250fdb7850d511aab83623d033acc575cb854f1346d6ccca1c6 Jul 15 21:04:35.103253 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:35.103221 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" event={"ID":"36b9ccbb-9fa2-4561-b234-49b301aeee60","Type":"ContainerStarted","Data":"6cdbe9f9d4ed7250fdb7850d511aab83623d033acc575cb854f1346d6ccca1c6"} Jul 15 21:04:35.104791 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:35.104758 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" event={"ID":"cc055e40-aa14-4337-8136-c1f7e5d51589","Type":"ContainerStarted","Data":"483c068db416aca756e02fd5a94e9ccb988da8a1f9abaa0bffef49a3318d40d0"} Jul 15 21:04:37.765904 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:37.765865 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:04:37.766210 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:37.765944 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:04:37.766210 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:37.765975 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:04:37.771332 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:37.771309 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:04:37.771385 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:37.771365 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:04:37.771425 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:37.771395 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:04:38.126591 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:38.126498 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" event={"ID":"cc055e40-aa14-4337-8136-c1f7e5d51589","Type":"ContainerStarted","Data":"a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d"} Jul 15 21:04:38.128402 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:38.128372 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" event={"ID":"36b9ccbb-9fa2-4561-b234-49b301aeee60","Type":"ContainerStarted","Data":"5ed324510b78d20e0cf52b5441b94f0242f8b34b5b5eed78ecc4c9dbc5638091"} Jul 15 21:04:38.191419 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:38.191361 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" podStartSLOduration=1.217074202 podStartE2EDuration="4.191335584s" totalImagesPullingTime="2.974261382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:04:38.158139742 +0000 UTC m=+331.248372921" watchObservedRunningTime="2026-07-15 21:04:38.191335584 +0000 UTC m=+331.281568764" Jul 15 21:04:38.191600 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:38.191511 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" podStartSLOduration=1.351428139 podStartE2EDuration="4.191506264s" totalImagesPullingTime="2.840078125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:04:38.188053363 +0000 UTC m=+331.278286568" watchObservedRunningTime="2026-07-15 21:04:38.191506264 +0000 UTC m=+331.281739448" Jul 15 21:04:38.636340 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:38.636304 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:38.637901 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:38.637873 2573 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 15 21:04:38.638041 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:38.637944 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" podUID="cc055e40-aa14-4337-8136-c1f7e5d51589" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 15 21:04:38.791861 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:38.791820 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:38.797311 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:38.797283 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:39.133908 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:39.133875 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:39.134633 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:39.134615 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46vn9s" Jul 15 21:04:39.211137 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:39.211107 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6"] Jul 15 21:04:39.636656 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:39.636619 2573 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 15 21:04:39.636839 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:39.636677 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" podUID="cc055e40-aa14-4337-8136-c1f7e5d51589" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 15 21:04:40.636522 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:40.636484 2573 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body= Jul 15 21:04:40.636987 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:40.636558 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" podUID="cc055e40-aa14-4337-8136-c1f7e5d51589" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" Jul 15 21:04:41.081574 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:41.081546 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-649f4f97b8-qpm5c" Jul 15 21:04:41.143384 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:41.143345 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" podUID="cc055e40-aa14-4337-8136-c1f7e5d51589" containerName="istio-proxy" containerID="cri-o://a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d" gracePeriod=30 Jul 15 21:04:43.095292 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:43.095250 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-ccf85" Jul 15 21:04:46.384996 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.384966 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:46.476310 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476224 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-certs\") pod \"cc055e40-aa14-4337-8136-c1f7e5d51589\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " Jul 15 21:04:46.476310 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476266 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-credential-socket\") pod \"cc055e40-aa14-4337-8136-c1f7e5d51589\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " Jul 15 21:04:46.476310 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476297 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-podinfo\") pod \"cc055e40-aa14-4337-8136-c1f7e5d51589\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " Jul 15 21:04:46.476580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476337 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-socket\") pod \"cc055e40-aa14-4337-8136-c1f7e5d51589\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " Jul 15 21:04:46.476580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476392 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-envoy\") pod \"cc055e40-aa14-4337-8136-c1f7e5d51589\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " Jul 15 21:04:46.476580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476415 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql7mz\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-kube-api-access-ql7mz\") pod \"cc055e40-aa14-4337-8136-c1f7e5d51589\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " Jul 15 21:04:46.476580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476479 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-data\") pod \"cc055e40-aa14-4337-8136-c1f7e5d51589\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " Jul 15 21:04:46.476580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476502 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-token\") pod \"cc055e40-aa14-4337-8136-c1f7e5d51589\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " Jul 15 21:04:46.476580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476512 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cc055e40-aa14-4337-8136-c1f7e5d51589" (UID: "cc055e40-aa14-4337-8136-c1f7e5d51589"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:04:46.476580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476565 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cc055e40-aa14-4337-8136-c1f7e5d51589" (UID: "cc055e40-aa14-4337-8136-c1f7e5d51589"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:04:46.476966 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476573 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc055e40-aa14-4337-8136-c1f7e5d51589-istiod-ca-cert\") pod \"cc055e40-aa14-4337-8136-c1f7e5d51589\" (UID: \"cc055e40-aa14-4337-8136-c1f7e5d51589\") " Jul 15 21:04:46.476966 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476631 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cc055e40-aa14-4337-8136-c1f7e5d51589" (UID: "cc055e40-aa14-4337-8136-c1f7e5d51589"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:04:46.476966 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476735 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-data" (OuterVolumeSpecName: "istio-data") pod "cc055e40-aa14-4337-8136-c1f7e5d51589" (UID: "cc055e40-aa14-4337-8136-c1f7e5d51589"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:04:46.476966 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476893 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc055e40-aa14-4337-8136-c1f7e5d51589-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cc055e40-aa14-4337-8136-c1f7e5d51589" (UID: "cc055e40-aa14-4337-8136-c1f7e5d51589"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:04:46.476966 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476933 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:04:46.476966 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476960 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:04:46.477282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476974 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:04:46.477282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.476988 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:04:46.478719 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.478688 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-kube-api-access-ql7mz" (OuterVolumeSpecName: "kube-api-access-ql7mz") pod "cc055e40-aa14-4337-8136-c1f7e5d51589" (UID: "cc055e40-aa14-4337-8136-c1f7e5d51589"). InnerVolumeSpecName "kube-api-access-ql7mz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:04:46.478830 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.478728 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-token" (OuterVolumeSpecName: "istio-token") pod "cc055e40-aa14-4337-8136-c1f7e5d51589" (UID: "cc055e40-aa14-4337-8136-c1f7e5d51589"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:04:46.478830 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.478750 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cc055e40-aa14-4337-8136-c1f7e5d51589" (UID: "cc055e40-aa14-4337-8136-c1f7e5d51589"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:04:46.478830 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.478757 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cc055e40-aa14-4337-8136-c1f7e5d51589" (UID: "cc055e40-aa14-4337-8136-c1f7e5d51589"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:04:46.578037 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.578005 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc055e40-aa14-4337-8136-c1f7e5d51589-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:04:46.578037 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.578032 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:04:46.578037 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.578041 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:04:46.578037 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.578050 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql7mz\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-kube-api-access-ql7mz\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:04:46.578405 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:46.578060 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc055e40-aa14-4337-8136-c1f7e5d51589-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:04:47.176254 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.176217 2573 generic.go:356] "Generic (PLEG): container finished" podID="cc055e40-aa14-4337-8136-c1f7e5d51589" containerID="a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d" exitCode=0 Jul 15 21:04:47.176446 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.176268 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" Jul 15 21:04:47.176446 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.176301 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" event={"ID":"cc055e40-aa14-4337-8136-c1f7e5d51589","Type":"ContainerDied","Data":"a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d"} Jul 15 21:04:47.176446 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.176354 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6" event={"ID":"cc055e40-aa14-4337-8136-c1f7e5d51589","Type":"ContainerDied","Data":"483c068db416aca756e02fd5a94e9ccb988da8a1f9abaa0bffef49a3318d40d0"} Jul 15 21:04:47.176446 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.176376 2573 scope.go:117] "RemoveContainer" containerID="a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d" Jul 15 21:04:47.186360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.186342 2573 scope.go:117] "RemoveContainer" containerID="a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d" Jul 15 21:04:47.186643 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:04:47.186622 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d\": container with ID starting with a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d not found: ID does not exist" containerID="a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d" Jul 15 21:04:47.186688 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.186651 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d"} err="failed to get container status \"a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d\": rpc error: code = NotFound desc = could not find container \"a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d\": container with ID starting with a953b98c03dc53de4c0fc5fe5072bcee197a68bc0d3ed7b68a6852571413d01d not found: ID does not exist" Jul 15 21:04:47.202979 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.202952 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6"] Jul 15 21:04:47.215920 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.215899 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-n5tk6"] Jul 15 21:04:47.448359 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:04:47.448276 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc055e40-aa14-4337-8136-c1f7e5d51589" path="/var/lib/kubelet/pods/cc055e40-aa14-4337-8136-c1f7e5d51589/volumes" Jul 15 21:05:27.502746 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.502710 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h8ppz"] Jul 15 21:05:27.503154 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.503053 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc055e40-aa14-4337-8136-c1f7e5d51589" containerName="istio-proxy" Jul 15 21:05:27.503154 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.503063 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc055e40-aa14-4337-8136-c1f7e5d51589" containerName="istio-proxy" Jul 15 21:05:27.503154 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.503131 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc055e40-aa14-4337-8136-c1f7e5d51589" containerName="istio-proxy" Jul 15 21:05:27.509400 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.509379 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" Jul 15 21:05:27.514533 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.514511 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 21:05:27.514650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.514557 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 21:05:27.518204 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.516879 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mctdj\"" Jul 15 21:05:27.519192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.519156 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h8ppz"] Jul 15 21:05:27.653905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.653874 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9nqg\" (UniqueName: \"kubernetes.io/projected/2b1f3412-eae3-4f5b-94c1-1eea70da4fcb-kube-api-access-c9nqg\") pod \"kuadrant-operator-catalog-h8ppz\" (UID: \"2b1f3412-eae3-4f5b-94c1-1eea70da4fcb\") " pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" Jul 15 21:05:27.754618 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.754591 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9nqg\" (UniqueName: \"kubernetes.io/projected/2b1f3412-eae3-4f5b-94c1-1eea70da4fcb-kube-api-access-c9nqg\") pod \"kuadrant-operator-catalog-h8ppz\" (UID: \"2b1f3412-eae3-4f5b-94c1-1eea70da4fcb\") " pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" Jul 15 21:05:27.764531 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.764499 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9nqg\" (UniqueName: \"kubernetes.io/projected/2b1f3412-eae3-4f5b-94c1-1eea70da4fcb-kube-api-access-c9nqg\") pod \"kuadrant-operator-catalog-h8ppz\" (UID: \"2b1f3412-eae3-4f5b-94c1-1eea70da4fcb\") " pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" Jul 15 21:05:27.820888 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.820847 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" Jul 15 21:05:27.849158 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.849128 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h8ppz"] Jul 15 21:05:27.949098 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:27.949075 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h8ppz"] Jul 15 21:05:27.951340 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:05:27.951311 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1f3412_eae3_4f5b_94c1_1eea70da4fcb.slice/crio-5cc917288626c8529d053df3461a5957a4c346c4d67a7d31d99371ff7c2231fe WatchSource:0}: Error finding container 5cc917288626c8529d053df3461a5957a4c346c4d67a7d31d99371ff7c2231fe: Status 404 returned error can't find the container with id 5cc917288626c8529d053df3461a5957a4c346c4d67a7d31d99371ff7c2231fe Jul 15 21:05:28.059061 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:28.058992 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-56hfv"] Jul 15 21:05:28.061950 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:28.061921 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-56hfv" Jul 15 21:05:28.073557 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:28.073533 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-56hfv"] Jul 15 21:05:28.158733 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:28.158699 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcq2v\" (UniqueName: \"kubernetes.io/projected/c86f5dcf-93c5-428a-90f7-3f7f710b3d5b-kube-api-access-gcq2v\") pod \"kuadrant-operator-catalog-56hfv\" (UID: \"c86f5dcf-93c5-428a-90f7-3f7f710b3d5b\") " pod="kuadrant-system/kuadrant-operator-catalog-56hfv" Jul 15 21:05:28.259293 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:28.259259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcq2v\" (UniqueName: \"kubernetes.io/projected/c86f5dcf-93c5-428a-90f7-3f7f710b3d5b-kube-api-access-gcq2v\") pod \"kuadrant-operator-catalog-56hfv\" (UID: \"c86f5dcf-93c5-428a-90f7-3f7f710b3d5b\") " pod="kuadrant-system/kuadrant-operator-catalog-56hfv" Jul 15 21:05:28.268899 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:28.268870 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcq2v\" (UniqueName: \"kubernetes.io/projected/c86f5dcf-93c5-428a-90f7-3f7f710b3d5b-kube-api-access-gcq2v\") pod \"kuadrant-operator-catalog-56hfv\" (UID: \"c86f5dcf-93c5-428a-90f7-3f7f710b3d5b\") " pod="kuadrant-system/kuadrant-operator-catalog-56hfv" Jul 15 21:05:28.371730 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:28.371627 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-56hfv" Jul 15 21:05:28.372919 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:28.372894 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" event={"ID":"2b1f3412-eae3-4f5b-94c1-1eea70da4fcb","Type":"ContainerStarted","Data":"5cc917288626c8529d053df3461a5957a4c346c4d67a7d31d99371ff7c2231fe"} Jul 15 21:05:28.526724 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:28.526689 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-56hfv"] Jul 15 21:05:28.528057 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:05:28.528029 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc86f5dcf_93c5_428a_90f7_3f7f710b3d5b.slice/crio-e8ddbf346c7e747993a335b472038e98e575db42c334b2165469c3b6d84160ef WatchSource:0}: Error finding container e8ddbf346c7e747993a335b472038e98e575db42c334b2165469c3b6d84160ef: Status 404 returned error can't find the container with id e8ddbf346c7e747993a335b472038e98e575db42c334b2165469c3b6d84160ef Jul 15 21:05:29.385106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:29.385061 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-56hfv" event={"ID":"c86f5dcf-93c5-428a-90f7-3f7f710b3d5b","Type":"ContainerStarted","Data":"e8ddbf346c7e747993a335b472038e98e575db42c334b2165469c3b6d84160ef"} Jul 15 21:05:30.392138 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.392099 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-56hfv" event={"ID":"c86f5dcf-93c5-428a-90f7-3f7f710b3d5b","Type":"ContainerStarted","Data":"53c9c4ea7059d48b077fbcaa8620c53fb7411115f27db8ed52fd0d81f680ddcd"} Jul 15 21:05:30.393788 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.393761 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" event={"ID":"2b1f3412-eae3-4f5b-94c1-1eea70da4fcb","Type":"ContainerStarted","Data":"8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c"} Jul 15 21:05:30.393924 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.393781 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" podUID="2b1f3412-eae3-4f5b-94c1-1eea70da4fcb" containerName="registry-server" containerID="cri-o://8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c" gracePeriod=2 Jul 15 21:05:30.412652 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.412607 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-56hfv" podStartSLOduration=0.721018126 podStartE2EDuration="2.412591875s" totalImagesPullingTime="1.691573749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:05:30.411211912 +0000 UTC m=+383.501445093" watchObservedRunningTime="2026-07-15 21:05:30.412591875 +0000 UTC m=+383.502825054" Jul 15 21:05:30.429350 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.429300 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" podStartSLOduration=1.16433078 podStartE2EDuration="3.42928315s" totalImagesPullingTime="2.26495237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:05:30.427935312 +0000 UTC m=+383.518168491" watchObservedRunningTime="2026-07-15 21:05:30.42928315 +0000 UTC m=+383.519516329" Jul 15 21:05:30.637935 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.637911 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h8ppz_2b1f3412-eae3-4f5b-94c1-1eea70da4fcb/registry-server/0.log" Jul 15 21:05:30.640241 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.640225 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" Jul 15 21:05:30.781836 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.781750 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9nqg\" (UniqueName: \"kubernetes.io/projected/2b1f3412-eae3-4f5b-94c1-1eea70da4fcb-kube-api-access-c9nqg\") pod \"2b1f3412-eae3-4f5b-94c1-1eea70da4fcb\" (UID: \"2b1f3412-eae3-4f5b-94c1-1eea70da4fcb\") " Jul 15 21:05:30.783939 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.783909 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b1f3412-eae3-4f5b-94c1-1eea70da4fcb-kube-api-access-c9nqg" (OuterVolumeSpecName: "kube-api-access-c9nqg") pod "2b1f3412-eae3-4f5b-94c1-1eea70da4fcb" (UID: "2b1f3412-eae3-4f5b-94c1-1eea70da4fcb"). InnerVolumeSpecName "kube-api-access-c9nqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:05:30.882635 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:30.882600 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9nqg\" (UniqueName: \"kubernetes.io/projected/2b1f3412-eae3-4f5b-94c1-1eea70da4fcb-kube-api-access-c9nqg\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:31.400001 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.399972 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h8ppz_2b1f3412-eae3-4f5b-94c1-1eea70da4fcb/registry-server/0.log" Jul 15 21:05:31.400429 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.400018 2573 generic.go:356] "Generic (PLEG): container finished" podID="2b1f3412-eae3-4f5b-94c1-1eea70da4fcb" containerID="8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c" exitCode=1 Jul 15 21:05:31.400429 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.400064 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" Jul 15 21:05:31.400429 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.400109 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" event={"ID":"2b1f3412-eae3-4f5b-94c1-1eea70da4fcb","Type":"ContainerDied","Data":"8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c"} Jul 15 21:05:31.400429 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.400156 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h8ppz" event={"ID":"2b1f3412-eae3-4f5b-94c1-1eea70da4fcb","Type":"ContainerDied","Data":"5cc917288626c8529d053df3461a5957a4c346c4d67a7d31d99371ff7c2231fe"} Jul 15 21:05:31.400429 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.400208 2573 scope.go:117] "RemoveContainer" containerID="8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c" Jul 15 21:05:31.409717 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.409700 2573 scope.go:117] "RemoveContainer" containerID="8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c" Jul 15 21:05:31.409945 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:05:31.409929 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c\": container with ID starting with 8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c not found: ID does not exist" containerID="8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c" Jul 15 21:05:31.410002 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.409952 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c"} err="failed to get container status \"8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c\": rpc error: code = NotFound desc = could not find container \"8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c\": container with ID starting with 8a354e3113d50e6050496d2f70ee3463f546ef8c6c401733c53f2490f4cfdf2c not found: ID does not exist" Jul 15 21:05:31.423315 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.423293 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h8ppz"] Jul 15 21:05:31.427435 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.427415 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h8ppz"] Jul 15 21:05:31.452049 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:31.452021 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b1f3412-eae3-4f5b-94c1-1eea70da4fcb" path="/var/lib/kubelet/pods/2b1f3412-eae3-4f5b-94c1-1eea70da4fcb/volumes" Jul 15 21:05:38.371799 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:38.371762 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-56hfv" Jul 15 21:05:38.372397 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:38.371852 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-56hfv" Jul 15 21:05:38.395132 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:38.395100 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-56hfv" Jul 15 21:05:38.455921 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:38.455889 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-56hfv" Jul 15 21:05:42.437747 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.437714 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9"] Jul 15 21:05:42.438137 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.438070 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b1f3412-eae3-4f5b-94c1-1eea70da4fcb" containerName="registry-server" Jul 15 21:05:42.438137 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.438081 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b1f3412-eae3-4f5b-94c1-1eea70da4fcb" containerName="registry-server" Jul 15 21:05:42.438238 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.438152 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b1f3412-eae3-4f5b-94c1-1eea70da4fcb" containerName="registry-server" Jul 15 21:05:42.442548 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.442529 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.445403 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.445386 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4qntx\"" Jul 15 21:05:42.450742 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.450719 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9"] Jul 15 21:05:42.595144 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.595096 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lqx8\" (UniqueName: \"kubernetes.io/projected/1348b943-daec-459c-9b72-cd854faa5e8b-kube-api-access-6lqx8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.595346 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.595160 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.595346 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.595221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.696028 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.695925 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lqx8\" (UniqueName: \"kubernetes.io/projected/1348b943-daec-459c-9b72-cd854faa5e8b-kube-api-access-6lqx8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.696028 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.695980 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.696296 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.696069 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.696437 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.696414 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.696478 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.696448 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.705876 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.705850 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lqx8\" (UniqueName: \"kubernetes.io/projected/1348b943-daec-459c-9b72-cd854faa5e8b-kube-api-access-6lqx8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.752687 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.752653 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:42.872474 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.872423 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz"] Jul 15 21:05:42.878483 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.878460 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:42.890604 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.890580 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz"] Jul 15 21:05:42.892880 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.892853 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9"] Jul 15 21:05:42.895067 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:05:42.895039 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1348b943_daec_459c_9b72_cd854faa5e8b.slice/crio-780300a2f5f485ab609d2ce0f6ebe5f23e41fad04ae60a3b3389f829dce03229 WatchSource:0}: Error finding container 780300a2f5f485ab609d2ce0f6ebe5f23e41fad04ae60a3b3389f829dce03229: Status 404 returned error can't find the container with id 780300a2f5f485ab609d2ce0f6ebe5f23e41fad04ae60a3b3389f829dce03229 Jul 15 21:05:42.897162 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.897136 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:42.897266 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.897233 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8thvz\" (UniqueName: \"kubernetes.io/projected/1fe444cd-bc64-49e6-a643-f175f8840a7c-kube-api-access-8thvz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:42.897333 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.897280 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:42.997999 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.997962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:42.998126 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.998074 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8thvz\" (UniqueName: \"kubernetes.io/projected/1fe444cd-bc64-49e6-a643-f175f8840a7c-kube-api-access-8thvz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:42.998207 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.998139 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:42.998382 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.998358 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:42.998459 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:42.998421 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:43.008255 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.008231 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8thvz\" (UniqueName: \"kubernetes.io/projected/1fe444cd-bc64-49e6-a643-f175f8840a7c-kube-api-access-8thvz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:43.187971 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.187939 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:43.322902 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.322874 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz"] Jul 15 21:05:43.325455 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:05:43.325429 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fe444cd_bc64_49e6_a643_f175f8840a7c.slice/crio-46e008cb67d2239bff705fe7b21ea2ce8b24a545a96ae816eaf6a132790fbcd4 WatchSource:0}: Error finding container 46e008cb67d2239bff705fe7b21ea2ce8b24a545a96ae816eaf6a132790fbcd4: Status 404 returned error can't find the container with id 46e008cb67d2239bff705fe7b21ea2ce8b24a545a96ae816eaf6a132790fbcd4 Jul 15 21:05:43.459325 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.459292 2573 generic.go:356] "Generic (PLEG): container finished" podID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerID="36b5fff4f33e3b8448f6b62936250dec6535279343ef9150deda37080971857f" exitCode=0 Jul 15 21:05:43.459745 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.459365 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" event={"ID":"1fe444cd-bc64-49e6-a643-f175f8840a7c","Type":"ContainerDied","Data":"36b5fff4f33e3b8448f6b62936250dec6535279343ef9150deda37080971857f"} Jul 15 21:05:43.459745 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.459411 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" event={"ID":"1fe444cd-bc64-49e6-a643-f175f8840a7c","Type":"ContainerStarted","Data":"46e008cb67d2239bff705fe7b21ea2ce8b24a545a96ae816eaf6a132790fbcd4"} Jul 15 21:05:43.460947 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.460922 2573 generic.go:356] "Generic (PLEG): container finished" podID="1348b943-daec-459c-9b72-cd854faa5e8b" containerID="6775092ba356548d28015ff23def3c5b6433df29500be756ea24f790d3627c4a" exitCode=0 Jul 15 21:05:43.461064 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.460969 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" event={"ID":"1348b943-daec-459c-9b72-cd854faa5e8b","Type":"ContainerDied","Data":"6775092ba356548d28015ff23def3c5b6433df29500be756ea24f790d3627c4a"} Jul 15 21:05:43.461064 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.461007 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" event={"ID":"1348b943-daec-459c-9b72-cd854faa5e8b","Type":"ContainerStarted","Data":"780300a2f5f485ab609d2ce0f6ebe5f23e41fad04ae60a3b3389f829dce03229"} Jul 15 21:05:43.469871 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.469849 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq"] Jul 15 21:05:43.473669 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.473651 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.483705 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.483677 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq"] Jul 15 21:05:43.603801 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.603770 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.603989 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.603849 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwssj\" (UniqueName: \"kubernetes.io/projected/97209923-879b-4976-a942-b09d8672b61f-kube-api-access-qwssj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.603989 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.603889 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.704580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.704532 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwssj\" (UniqueName: \"kubernetes.io/projected/97209923-879b-4976-a942-b09d8672b61f-kube-api-access-qwssj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.704742 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.704677 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.704742 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.704706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.704998 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.704979 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.705063 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.705046 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.713131 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.713110 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwssj\" (UniqueName: \"kubernetes.io/projected/97209923-879b-4976-a942-b09d8672b61f-kube-api-access-qwssj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.783106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.783071 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:43.916903 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:43.916876 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq"] Jul 15 21:05:43.918581 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:05:43.918553 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97209923_879b_4976_a942_b09d8672b61f.slice/crio-1236de0b5446a4de345f5fe985a7b4dbc7ce4756f35a1a2a2d326e383c1edf5e WatchSource:0}: Error finding container 1236de0b5446a4de345f5fe985a7b4dbc7ce4756f35a1a2a2d326e383c1edf5e: Status 404 returned error can't find the container with id 1236de0b5446a4de345f5fe985a7b4dbc7ce4756f35a1a2a2d326e383c1edf5e Jul 15 21:05:44.056688 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.056657 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q"] Jul 15 21:05:44.061145 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.061122 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.071370 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.071339 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q"] Jul 15 21:05:44.107416 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.107384 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tqkh\" (UniqueName: \"kubernetes.io/projected/9450047d-2a40-4ea7-a887-9dbe4c48269d-kube-api-access-5tqkh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.107556 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.107421 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.107556 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.107514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.208707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.208680 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tqkh\" (UniqueName: \"kubernetes.io/projected/9450047d-2a40-4ea7-a887-9dbe4c48269d-kube-api-access-5tqkh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.208833 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.208716 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.208833 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.208772 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.209197 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.209161 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.209197 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.209190 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.218747 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.218729 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tqkh\" (UniqueName: \"kubernetes.io/projected/9450047d-2a40-4ea7-a887-9dbe4c48269d-kube-api-access-5tqkh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.393424 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.393393 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:44.468308 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.468271 2573 generic.go:356] "Generic (PLEG): container finished" podID="97209923-879b-4976-a942-b09d8672b61f" containerID="81d53d387a8adb30aea87226a13a859fc25de94b85ff7ed9fbb0ead4f49e0d41" exitCode=0 Jul 15 21:05:44.468705 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.468315 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" event={"ID":"97209923-879b-4976-a942-b09d8672b61f","Type":"ContainerDied","Data":"81d53d387a8adb30aea87226a13a859fc25de94b85ff7ed9fbb0ead4f49e0d41"} Jul 15 21:05:44.468705 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.468372 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" event={"ID":"97209923-879b-4976-a942-b09d8672b61f","Type":"ContainerStarted","Data":"1236de0b5446a4de345f5fe985a7b4dbc7ce4756f35a1a2a2d326e383c1edf5e"} Jul 15 21:05:44.470463 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.470440 2573 generic.go:356] "Generic (PLEG): container finished" podID="1348b943-daec-459c-9b72-cd854faa5e8b" containerID="8661d69eb851ccd61bb284ee3b6199408eef5e985910f66c922308ab58d40bbc" exitCode=0 Jul 15 21:05:44.470581 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.470475 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" event={"ID":"1348b943-daec-459c-9b72-cd854faa5e8b","Type":"ContainerDied","Data":"8661d69eb851ccd61bb284ee3b6199408eef5e985910f66c922308ab58d40bbc"} Jul 15 21:05:44.533342 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:44.533235 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q"] Jul 15 21:05:44.535725 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:05:44.535706 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9450047d_2a40_4ea7_a887_9dbe4c48269d.slice/crio-64a76326d74bdf10b12a9642510e3a490912a0d9b7dfdfed8b8bb0c28af0e1fd WatchSource:0}: Error finding container 64a76326d74bdf10b12a9642510e3a490912a0d9b7dfdfed8b8bb0c28af0e1fd: Status 404 returned error can't find the container with id 64a76326d74bdf10b12a9642510e3a490912a0d9b7dfdfed8b8bb0c28af0e1fd Jul 15 21:05:45.478295 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:45.478263 2573 generic.go:356] "Generic (PLEG): container finished" podID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerID="985b63f20899d8e202f642143af4b5f4112942b35f17e0c4d2de0bde78c7ba07" exitCode=0 Jul 15 21:05:45.478661 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:45.478352 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" event={"ID":"1fe444cd-bc64-49e6-a643-f175f8840a7c","Type":"ContainerDied","Data":"985b63f20899d8e202f642143af4b5f4112942b35f17e0c4d2de0bde78c7ba07"} Jul 15 21:05:45.480495 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:45.480464 2573 generic.go:356] "Generic (PLEG): container finished" podID="1348b943-daec-459c-9b72-cd854faa5e8b" containerID="48f5dfe7a15177278e45f0286db1e795405d0d6ee768dd10434875b13629de8b" exitCode=0 Jul 15 21:05:45.480592 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:45.480492 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" event={"ID":"1348b943-daec-459c-9b72-cd854faa5e8b","Type":"ContainerDied","Data":"48f5dfe7a15177278e45f0286db1e795405d0d6ee768dd10434875b13629de8b"} Jul 15 21:05:45.481866 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:45.481848 2573 generic.go:356] "Generic (PLEG): container finished" podID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerID="0d5fc39e876acb263850edb52b2cc6eabad77c2758e87e6719893f62b4779df6" exitCode=0 Jul 15 21:05:45.481958 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:45.481926 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" event={"ID":"9450047d-2a40-4ea7-a887-9dbe4c48269d","Type":"ContainerDied","Data":"0d5fc39e876acb263850edb52b2cc6eabad77c2758e87e6719893f62b4779df6"} Jul 15 21:05:45.482022 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:45.481970 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" event={"ID":"9450047d-2a40-4ea7-a887-9dbe4c48269d","Type":"ContainerStarted","Data":"64a76326d74bdf10b12a9642510e3a490912a0d9b7dfdfed8b8bb0c28af0e1fd"} Jul 15 21:05:45.483790 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:45.483770 2573 generic.go:356] "Generic (PLEG): container finished" podID="97209923-879b-4976-a942-b09d8672b61f" containerID="5682ec3d759a56c3bd7561cb1501b11a9e604fc4f70ff9fa2fdb354bbc643af7" exitCode=0 Jul 15 21:05:45.483921 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:45.483819 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" event={"ID":"97209923-879b-4976-a942-b09d8672b61f","Type":"ContainerDied","Data":"5682ec3d759a56c3bd7561cb1501b11a9e604fc4f70ff9fa2fdb354bbc643af7"} Jul 15 21:05:46.491403 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.491368 2573 generic.go:356] "Generic (PLEG): container finished" podID="97209923-879b-4976-a942-b09d8672b61f" containerID="fea12d04ea1946435eb9e34053b59cd23e2a80ec0f6b01bbe52c916a13acdd7d" exitCode=0 Jul 15 21:05:46.491808 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.491449 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" event={"ID":"97209923-879b-4976-a942-b09d8672b61f","Type":"ContainerDied","Data":"fea12d04ea1946435eb9e34053b59cd23e2a80ec0f6b01bbe52c916a13acdd7d"} Jul 15 21:05:46.493462 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.493441 2573 generic.go:356] "Generic (PLEG): container finished" podID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerID="c369efccc6ad0660f2b4985db18ddfb4ac4eeb3d70e55107b88313843179fa9b" exitCode=0 Jul 15 21:05:46.493547 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.493470 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" event={"ID":"1fe444cd-bc64-49e6-a643-f175f8840a7c","Type":"ContainerDied","Data":"c369efccc6ad0660f2b4985db18ddfb4ac4eeb3d70e55107b88313843179fa9b"} Jul 15 21:05:46.625341 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.625315 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:46.627734 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.627716 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-util\") pod \"1348b943-daec-459c-9b72-cd854faa5e8b\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " Jul 15 21:05:46.627789 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.627776 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lqx8\" (UniqueName: \"kubernetes.io/projected/1348b943-daec-459c-9b72-cd854faa5e8b-kube-api-access-6lqx8\") pod \"1348b943-daec-459c-9b72-cd854faa5e8b\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " Jul 15 21:05:46.627920 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.627904 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-bundle\") pod \"1348b943-daec-459c-9b72-cd854faa5e8b\" (UID: \"1348b943-daec-459c-9b72-cd854faa5e8b\") " Jul 15 21:05:46.628450 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.628427 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-bundle" (OuterVolumeSpecName: "bundle") pod "1348b943-daec-459c-9b72-cd854faa5e8b" (UID: "1348b943-daec-459c-9b72-cd854faa5e8b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:05:46.629830 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.629806 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1348b943-daec-459c-9b72-cd854faa5e8b-kube-api-access-6lqx8" (OuterVolumeSpecName: "kube-api-access-6lqx8") pod "1348b943-daec-459c-9b72-cd854faa5e8b" (UID: "1348b943-daec-459c-9b72-cd854faa5e8b"). InnerVolumeSpecName "kube-api-access-6lqx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:05:46.636053 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.636008 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-util" (OuterVolumeSpecName: "util") pod "1348b943-daec-459c-9b72-cd854faa5e8b" (UID: "1348b943-daec-459c-9b72-cd854faa5e8b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:05:46.729660 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.729626 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-bundle\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:46.729660 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.729656 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1348b943-daec-459c-9b72-cd854faa5e8b-util\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:46.729660 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:46.729665 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lqx8\" (UniqueName: \"kubernetes.io/projected/1348b943-daec-459c-9b72-cd854faa5e8b-kube-api-access-6lqx8\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:47.500555 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.500526 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" Jul 15 21:05:47.500956 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.500529 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9" event={"ID":"1348b943-daec-459c-9b72-cd854faa5e8b","Type":"ContainerDied","Data":"780300a2f5f485ab609d2ce0f6ebe5f23e41fad04ae60a3b3389f829dce03229"} Jul 15 21:05:47.500956 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.500643 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="780300a2f5f485ab609d2ce0f6ebe5f23e41fad04ae60a3b3389f829dce03229" Jul 15 21:05:47.502345 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.502315 2573 generic.go:356] "Generic (PLEG): container finished" podID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerID="860324c14a1d21de3a0b0df0bf125b7a34af176297857370ec09c62530842d90" exitCode=0 Jul 15 21:05:47.502463 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.502389 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" event={"ID":"9450047d-2a40-4ea7-a887-9dbe4c48269d","Type":"ContainerDied","Data":"860324c14a1d21de3a0b0df0bf125b7a34af176297857370ec09c62530842d90"} Jul 15 21:05:47.652256 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.652231 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:47.655937 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.655894 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:47.737908 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.737877 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-bundle\") pod \"97209923-879b-4976-a942-b09d8672b61f\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " Jul 15 21:05:47.738070 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.737921 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-util\") pod \"1fe444cd-bc64-49e6-a643-f175f8840a7c\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " Jul 15 21:05:47.738070 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.737961 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8thvz\" (UniqueName: \"kubernetes.io/projected/1fe444cd-bc64-49e6-a643-f175f8840a7c-kube-api-access-8thvz\") pod \"1fe444cd-bc64-49e6-a643-f175f8840a7c\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " Jul 15 21:05:47.738070 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.738002 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-util\") pod \"97209923-879b-4976-a942-b09d8672b61f\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " Jul 15 21:05:47.738070 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.738041 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-bundle\") pod \"1fe444cd-bc64-49e6-a643-f175f8840a7c\" (UID: \"1fe444cd-bc64-49e6-a643-f175f8840a7c\") " Jul 15 21:05:47.738331 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.738195 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwssj\" (UniqueName: \"kubernetes.io/projected/97209923-879b-4976-a942-b09d8672b61f-kube-api-access-qwssj\") pod \"97209923-879b-4976-a942-b09d8672b61f\" (UID: \"97209923-879b-4976-a942-b09d8672b61f\") " Jul 15 21:05:47.738511 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.738486 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-bundle" (OuterVolumeSpecName: "bundle") pod "1fe444cd-bc64-49e6-a643-f175f8840a7c" (UID: "1fe444cd-bc64-49e6-a643-f175f8840a7c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:05:47.738625 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.738532 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-bundle" (OuterVolumeSpecName: "bundle") pod "97209923-879b-4976-a942-b09d8672b61f" (UID: "97209923-879b-4976-a942-b09d8672b61f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:05:47.740508 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.740486 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97209923-879b-4976-a942-b09d8672b61f-kube-api-access-qwssj" (OuterVolumeSpecName: "kube-api-access-qwssj") pod "97209923-879b-4976-a942-b09d8672b61f" (UID: "97209923-879b-4976-a942-b09d8672b61f"). InnerVolumeSpecName "kube-api-access-qwssj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:05:47.740641 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.740622 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fe444cd-bc64-49e6-a643-f175f8840a7c-kube-api-access-8thvz" (OuterVolumeSpecName: "kube-api-access-8thvz") pod "1fe444cd-bc64-49e6-a643-f175f8840a7c" (UID: "1fe444cd-bc64-49e6-a643-f175f8840a7c"). InnerVolumeSpecName "kube-api-access-8thvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:05:47.744687 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.744655 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-util" (OuterVolumeSpecName: "util") pod "1fe444cd-bc64-49e6-a643-f175f8840a7c" (UID: "1fe444cd-bc64-49e6-a643-f175f8840a7c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:05:47.744832 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.744814 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-util" (OuterVolumeSpecName: "util") pod "97209923-879b-4976-a942-b09d8672b61f" (UID: "97209923-879b-4976-a942-b09d8672b61f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:05:47.838955 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.838870 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8thvz\" (UniqueName: \"kubernetes.io/projected/1fe444cd-bc64-49e6-a643-f175f8840a7c-kube-api-access-8thvz\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:47.838955 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.838899 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-util\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:47.838955 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.838908 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-bundle\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:47.838955 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.838916 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwssj\" (UniqueName: \"kubernetes.io/projected/97209923-879b-4976-a942-b09d8672b61f-kube-api-access-qwssj\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:47.838955 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.838925 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97209923-879b-4976-a942-b09d8672b61f-bundle\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:47.838955 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:47.838933 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1fe444cd-bc64-49e6-a643-f175f8840a7c-util\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:48.509542 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:48.509509 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" Jul 15 21:05:48.509542 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:48.509529 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq" event={"ID":"97209923-879b-4976-a942-b09d8672b61f","Type":"ContainerDied","Data":"1236de0b5446a4de345f5fe985a7b4dbc7ce4756f35a1a2a2d326e383c1edf5e"} Jul 15 21:05:48.509982 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:48.509577 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1236de0b5446a4de345f5fe985a7b4dbc7ce4756f35a1a2a2d326e383c1edf5e" Jul 15 21:05:48.511506 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:48.511484 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" Jul 15 21:05:48.511506 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:48.511499 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz" event={"ID":"1fe444cd-bc64-49e6-a643-f175f8840a7c","Type":"ContainerDied","Data":"46e008cb67d2239bff705fe7b21ea2ce8b24a545a96ae816eaf6a132790fbcd4"} Jul 15 21:05:48.511677 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:48.511534 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46e008cb67d2239bff705fe7b21ea2ce8b24a545a96ae816eaf6a132790fbcd4" Jul 15 21:05:48.513860 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:48.513836 2573 generic.go:356] "Generic (PLEG): container finished" podID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerID="37f98a42f4f2672939621c15b37e41b181920c4c158755f0e45dca130f4b2ef3" exitCode=0 Jul 15 21:05:48.513954 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:48.513904 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" event={"ID":"9450047d-2a40-4ea7-a887-9dbe4c48269d","Type":"ContainerDied","Data":"37f98a42f4f2672939621c15b37e41b181920c4c158755f0e45dca130f4b2ef3"} Jul 15 21:05:48.549453 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:05:48.549418 2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97209923_879b_4976_a942_b09d8672b61f.slice\": RecentStats: unable to find data in memory cache]" Jul 15 21:05:49.653943 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.653915 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:49.754086 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.754047 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-util\") pod \"9450047d-2a40-4ea7-a887-9dbe4c48269d\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " Jul 15 21:05:49.754282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.754110 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tqkh\" (UniqueName: \"kubernetes.io/projected/9450047d-2a40-4ea7-a887-9dbe4c48269d-kube-api-access-5tqkh\") pod \"9450047d-2a40-4ea7-a887-9dbe4c48269d\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " Jul 15 21:05:49.754282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.754139 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-bundle\") pod \"9450047d-2a40-4ea7-a887-9dbe4c48269d\" (UID: \"9450047d-2a40-4ea7-a887-9dbe4c48269d\") " Jul 15 21:05:49.754659 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.754638 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-bundle" (OuterVolumeSpecName: "bundle") pod "9450047d-2a40-4ea7-a887-9dbe4c48269d" (UID: "9450047d-2a40-4ea7-a887-9dbe4c48269d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:05:49.756309 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.756288 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9450047d-2a40-4ea7-a887-9dbe4c48269d-kube-api-access-5tqkh" (OuterVolumeSpecName: "kube-api-access-5tqkh") pod "9450047d-2a40-4ea7-a887-9dbe4c48269d" (UID: "9450047d-2a40-4ea7-a887-9dbe4c48269d"). InnerVolumeSpecName "kube-api-access-5tqkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:05:49.846347 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.846263 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-util" (OuterVolumeSpecName: "util") pod "9450047d-2a40-4ea7-a887-9dbe4c48269d" (UID: "9450047d-2a40-4ea7-a887-9dbe4c48269d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:05:49.855714 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.855693 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tqkh\" (UniqueName: \"kubernetes.io/projected/9450047d-2a40-4ea7-a887-9dbe4c48269d-kube-api-access-5tqkh\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:49.855714 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.855714 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-bundle\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:49.855816 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:49.855723 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9450047d-2a40-4ea7-a887-9dbe4c48269d-util\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:05:50.526369 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:50.526341 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" Jul 15 21:05:50.526369 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:50.526354 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q" event={"ID":"9450047d-2a40-4ea7-a887-9dbe4c48269d","Type":"ContainerDied","Data":"64a76326d74bdf10b12a9642510e3a490912a0d9b7dfdfed8b8bb0c28af0e1fd"} Jul 15 21:05:50.526615 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:05:50.526394 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64a76326d74bdf10b12a9642510e3a490912a0d9b7dfdfed8b8bb0c28af0e1fd" Jul 15 21:06:00.609045 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609011 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k"] Jul 15 21:06:00.609450 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609409 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerName="extract" Jul 15 21:06:00.609450 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609421 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerName="extract" Jul 15 21:06:00.609450 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609429 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1348b943-daec-459c-9b72-cd854faa5e8b" containerName="pull" Jul 15 21:06:00.609450 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609435 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1348b943-daec-459c-9b72-cd854faa5e8b" containerName="pull" Jul 15 21:06:00.609450 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609447 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerName="pull" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609456 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerName="pull" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609471 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerName="extract" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609476 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerName="extract" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609485 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerName="util" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609489 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerName="util" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609495 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerName="pull" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609499 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerName="pull" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609508 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97209923-879b-4976-a942-b09d8672b61f" containerName="util" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609512 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="97209923-879b-4976-a942-b09d8672b61f" containerName="util" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609517 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerName="util" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609522 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerName="util" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609529 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1348b943-daec-459c-9b72-cd854faa5e8b" containerName="extract" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609537 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1348b943-daec-459c-9b72-cd854faa5e8b" containerName="extract" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609546 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97209923-879b-4976-a942-b09d8672b61f" containerName="extract" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609551 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="97209923-879b-4976-a942-b09d8672b61f" containerName="extract" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609558 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1348b943-daec-459c-9b72-cd854faa5e8b" containerName="util" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609563 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1348b943-daec-459c-9b72-cd854faa5e8b" containerName="util" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609568 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97209923-879b-4976-a942-b09d8672b61f" containerName="pull" Jul 15 21:06:00.609601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609572 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="97209923-879b-4976-a942-b09d8672b61f" containerName="pull" Jul 15 21:06:00.610106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609637 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9450047d-2a40-4ea7-a887-9dbe4c48269d" containerName="extract" Jul 15 21:06:00.610106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609646 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="97209923-879b-4976-a942-b09d8672b61f" containerName="extract" Jul 15 21:06:00.610106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609653 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1348b943-daec-459c-9b72-cd854faa5e8b" containerName="extract" Jul 15 21:06:00.610106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.609659 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fe444cd-bc64-49e6-a643-f175f8840a7c" containerName="extract" Jul 15 21:06:00.612469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.612442 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:00.616122 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.616102 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pcsdv\"" Jul 15 21:06:00.641692 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.641667 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k"] Jul 15 21:06:00.648407 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.648385 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425eee9b-44fc-4022-8e45-832e31718a15-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" (UID: \"425eee9b-44fc-4022-8e45-832e31718a15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:00.648525 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.648426 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptq9g\" (UniqueName: \"kubernetes.io/projected/425eee9b-44fc-4022-8e45-832e31718a15-kube-api-access-ptq9g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" (UID: \"425eee9b-44fc-4022-8e45-832e31718a15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:00.749061 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.749018 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425eee9b-44fc-4022-8e45-832e31718a15-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" (UID: \"425eee9b-44fc-4022-8e45-832e31718a15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:00.749240 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.749072 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptq9g\" (UniqueName: \"kubernetes.io/projected/425eee9b-44fc-4022-8e45-832e31718a15-kube-api-access-ptq9g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" (UID: \"425eee9b-44fc-4022-8e45-832e31718a15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:00.749527 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.749506 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425eee9b-44fc-4022-8e45-832e31718a15-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" (UID: \"425eee9b-44fc-4022-8e45-832e31718a15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:00.776283 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.776253 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptq9g\" (UniqueName: \"kubernetes.io/projected/425eee9b-44fc-4022-8e45-832e31718a15-kube-api-access-ptq9g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" (UID: \"425eee9b-44fc-4022-8e45-832e31718a15\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:00.922729 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:00.922642 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:01.086935 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:01.086907 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k"] Jul 15 21:06:01.088461 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:06:01.088433 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod425eee9b_44fc_4022_8e45_832e31718a15.slice/crio-ce3f0556c2388e9e18a8efb584b6ed6ce1a55893ab45e06a226b27799b3bba3f WatchSource:0}: Error finding container ce3f0556c2388e9e18a8efb584b6ed6ce1a55893ab45e06a226b27799b3bba3f: Status 404 returned error can't find the container with id ce3f0556c2388e9e18a8efb584b6ed6ce1a55893ab45e06a226b27799b3bba3f Jul 15 21:06:01.590166 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:01.590132 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" event={"ID":"425eee9b-44fc-4022-8e45-832e31718a15","Type":"ContainerStarted","Data":"ce3f0556c2388e9e18a8efb584b6ed6ce1a55893ab45e06a226b27799b3bba3f"} Jul 15 21:06:02.605562 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:02.605530 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59"] Jul 15 21:06:02.609646 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:02.609625 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" Jul 15 21:06:02.612550 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:02.612521 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-mn2pv\"" Jul 15 21:06:02.612550 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:02.612541 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 15 21:06:02.621968 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:02.621949 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59"] Jul 15 21:06:02.668031 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:02.668002 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzgwn\" (UniqueName: \"kubernetes.io/projected/01f63185-f7d1-4781-b61a-58705dc8439b-kube-api-access-vzgwn\") pod \"dns-operator-controller-manager-864dd78c4c-89h59\" (UID: \"01f63185-f7d1-4781-b61a-58705dc8439b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" Jul 15 21:06:02.769143 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:02.769098 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzgwn\" (UniqueName: \"kubernetes.io/projected/01f63185-f7d1-4781-b61a-58705dc8439b-kube-api-access-vzgwn\") pod \"dns-operator-controller-manager-864dd78c4c-89h59\" (UID: \"01f63185-f7d1-4781-b61a-58705dc8439b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" Jul 15 21:06:02.778785 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:02.778754 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzgwn\" (UniqueName: \"kubernetes.io/projected/01f63185-f7d1-4781-b61a-58705dc8439b-kube-api-access-vzgwn\") pod \"dns-operator-controller-manager-864dd78c4c-89h59\" (UID: \"01f63185-f7d1-4781-b61a-58705dc8439b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" Jul 15 21:06:02.920371 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:02.920286 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" Jul 15 21:06:03.066495 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:03.061801 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59"] Jul 15 21:06:03.602385 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:03.602352 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" event={"ID":"01f63185-f7d1-4781-b61a-58705dc8439b","Type":"ContainerStarted","Data":"893315208ef59f2ec0021de2ba82c25e3bdceef9a4a6902c9398d53e88bc304b"} Jul 15 21:06:06.621954 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:06.621910 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" event={"ID":"01f63185-f7d1-4781-b61a-58705dc8439b","Type":"ContainerStarted","Data":"2ca665b10aa8c8363dee54f1ddc38bf2da4931dec512fa59e3ba9a6b05e87243"} Jul 15 21:06:06.622442 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:06.621990 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" Jul 15 21:06:06.644930 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:06.644881 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" podStartSLOduration=2.037257199 podStartE2EDuration="4.644861894s" totalImagesPullingTime="2.607604695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:06.641497463 +0000 UTC m=+419.731730641" watchObservedRunningTime="2026-07-15 21:06:06.644861894 +0000 UTC m=+419.735095073" Jul 15 21:06:09.645659 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:09.645564 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" event={"ID":"425eee9b-44fc-4022-8e45-832e31718a15","Type":"ContainerStarted","Data":"6d2bddc55aa52137c91247d63caff6e334fc5978dba6e6900e62d68f1a99b39e"} Jul 15 21:06:09.646107 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:09.645691 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:09.674887 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:09.674831 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" podStartSLOduration=1.371350555 podStartE2EDuration="9.674813412s" totalImagesPullingTime="8.303462857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:09.671461614 +0000 UTC m=+422.761694791" watchObservedRunningTime="2026-07-15 21:06:09.674813412 +0000 UTC m=+422.765046592" Jul 15 21:06:17.630083 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:17.630055 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-89h59" Jul 15 21:06:20.653020 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:20.652983 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:22.575552 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.575518 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k"] Jul 15 21:06:22.575951 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.575708 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" podUID="425eee9b-44fc-4022-8e45-832e31718a15" containerName="manager" containerID="cri-o://6d2bddc55aa52137c91247d63caff6e334fc5978dba6e6900e62d68f1a99b39e" gracePeriod=2 Jul 15 21:06:22.587998 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.587968 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k"] Jul 15 21:06:22.592744 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:06:22.592711 2573 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"425eee9b-44fc-4022-8e45-832e31718a15\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\": pods \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:22.609306 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.609280 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn"] Jul 15 21:06:22.609706 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.609694 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="425eee9b-44fc-4022-8e45-832e31718a15" containerName="manager" Jul 15 21:06:22.609746 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.609709 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="425eee9b-44fc-4022-8e45-832e31718a15" containerName="manager" Jul 15 21:06:22.609796 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.609785 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="425eee9b-44fc-4022-8e45-832e31718a15" containerName="manager" Jul 15 21:06:22.615459 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.615439 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:22.618421 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:06:22.618389 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="425eee9b-44fc-4022-8e45-832e31718a15" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:22.630072 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.630050 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn"] Jul 15 21:06:22.717107 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.717075 2573 generic.go:356] "Generic (PLEG): container finished" podID="425eee9b-44fc-4022-8e45-832e31718a15" containerID="6d2bddc55aa52137c91247d63caff6e334fc5978dba6e6900e62d68f1a99b39e" exitCode=0 Jul 15 21:06:22.755757 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.755730 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n58t4\" (UniqueName: \"kubernetes.io/projected/a5606e5b-d672-4f25-9f88-7910e18bf2ba-kube-api-access-n58t4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mtzdn\" (UID: \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:22.755894 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.755804 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5606e5b-d672-4f25-9f88-7910e18bf2ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mtzdn\" (UID: \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:22.793051 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.793022 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm"] Jul 15 21:06:22.796633 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.796481 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:22.799075 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:06:22.799053 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="425eee9b-44fc-4022-8e45-832e31718a15" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:22.815324 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.815296 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm"] Jul 15 21:06:22.819657 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.819635 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:22.835401 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:06:22.835328 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8dv4k\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="425eee9b-44fc-4022-8e45-832e31718a15" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:22.856362 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.856333 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5606e5b-d672-4f25-9f88-7910e18bf2ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mtzdn\" (UID: \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:22.856484 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.856423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n58t4\" (UniqueName: \"kubernetes.io/projected/a5606e5b-d672-4f25-9f88-7910e18bf2ba-kube-api-access-n58t4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mtzdn\" (UID: \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:22.856739 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.856720 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5606e5b-d672-4f25-9f88-7910e18bf2ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mtzdn\" (UID: \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:22.867658 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.867631 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n58t4\" (UniqueName: \"kubernetes.io/projected/a5606e5b-d672-4f25-9f88-7910e18bf2ba-kube-api-access-n58t4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mtzdn\" (UID: \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:22.953298 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.953249 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:22.957345 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.957321 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425eee9b-44fc-4022-8e45-832e31718a15-extensions-socket-volume\") pod \"425eee9b-44fc-4022-8e45-832e31718a15\" (UID: \"425eee9b-44fc-4022-8e45-832e31718a15\") " Jul 15 21:06:22.957423 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.957356 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptq9g\" (UniqueName: \"kubernetes.io/projected/425eee9b-44fc-4022-8e45-832e31718a15-kube-api-access-ptq9g\") pod \"425eee9b-44fc-4022-8e45-832e31718a15\" (UID: \"425eee9b-44fc-4022-8e45-832e31718a15\") " Jul 15 21:06:22.957591 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.957570 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3373f38d-efe1-4caf-9673-6663cfb5713c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nm2dm\" (UID: \"3373f38d-efe1-4caf-9673-6663cfb5713c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:22.957670 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.957655 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfzn7\" (UniqueName: \"kubernetes.io/projected/3373f38d-efe1-4caf-9673-6663cfb5713c-kube-api-access-jfzn7\") pod \"kuadrant-operator-controller-manager-86878c769f-nm2dm\" (UID: \"3373f38d-efe1-4caf-9673-6663cfb5713c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:22.957831 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.957812 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/425eee9b-44fc-4022-8e45-832e31718a15-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "425eee9b-44fc-4022-8e45-832e31718a15" (UID: "425eee9b-44fc-4022-8e45-832e31718a15"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:06:22.959542 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:22.959516 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/425eee9b-44fc-4022-8e45-832e31718a15-kube-api-access-ptq9g" (OuterVolumeSpecName: "kube-api-access-ptq9g") pod "425eee9b-44fc-4022-8e45-832e31718a15" (UID: "425eee9b-44fc-4022-8e45-832e31718a15"). InnerVolumeSpecName "kube-api-access-ptq9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:06:23.058906 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.058865 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3373f38d-efe1-4caf-9673-6663cfb5713c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nm2dm\" (UID: \"3373f38d-efe1-4caf-9673-6663cfb5713c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:23.059074 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.058976 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfzn7\" (UniqueName: \"kubernetes.io/projected/3373f38d-efe1-4caf-9673-6663cfb5713c-kube-api-access-jfzn7\") pod \"kuadrant-operator-controller-manager-86878c769f-nm2dm\" (UID: \"3373f38d-efe1-4caf-9673-6663cfb5713c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:23.059074 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.059056 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/425eee9b-44fc-4022-8e45-832e31718a15-extensions-socket-volume\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:06:23.059074 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.059071 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptq9g\" (UniqueName: \"kubernetes.io/projected/425eee9b-44fc-4022-8e45-832e31718a15-kube-api-access-ptq9g\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:06:23.059300 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.059274 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3373f38d-efe1-4caf-9673-6663cfb5713c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nm2dm\" (UID: \"3373f38d-efe1-4caf-9673-6663cfb5713c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:23.079396 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.079371 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfzn7\" (UniqueName: \"kubernetes.io/projected/3373f38d-efe1-4caf-9673-6663cfb5713c-kube-api-access-jfzn7\") pod \"kuadrant-operator-controller-manager-86878c769f-nm2dm\" (UID: \"3373f38d-efe1-4caf-9673-6663cfb5713c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:23.102313 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.102278 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn"] Jul 15 21:06:23.104814 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:06:23.104781 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5606e5b_d672_4f25_9f88_7910e18bf2ba.slice/crio-aa90323544f949517e244d96340c461a16b0634f218574db1a8bf68ed6da5d18 WatchSource:0}: Error finding container aa90323544f949517e244d96340c461a16b0634f218574db1a8bf68ed6da5d18: Status 404 returned error can't find the container with id aa90323544f949517e244d96340c461a16b0634f218574db1a8bf68ed6da5d18 Jul 15 21:06:23.125070 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.125047 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:23.279588 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.279565 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm"] Jul 15 21:06:23.281321 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:06:23.281296 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3373f38d_efe1_4caf_9673_6663cfb5713c.slice/crio-dc18963286c2c5dc2e0e5c9133508668ad4ee497e38f736106bd75c6d6c38025 WatchSource:0}: Error finding container dc18963286c2c5dc2e0e5c9133508668ad4ee497e38f736106bd75c6d6c38025: Status 404 returned error can't find the container with id dc18963286c2c5dc2e0e5c9133508668ad4ee497e38f736106bd75c6d6c38025 Jul 15 21:06:23.448032 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.447960 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="425eee9b-44fc-4022-8e45-832e31718a15" path="/var/lib/kubelet/pods/425eee9b-44fc-4022-8e45-832e31718a15/volumes" Jul 15 21:06:23.723768 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.723685 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" event={"ID":"a5606e5b-d672-4f25-9f88-7910e18bf2ba","Type":"ContainerStarted","Data":"435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878"} Jul 15 21:06:23.723768 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.723745 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" event={"ID":"a5606e5b-d672-4f25-9f88-7910e18bf2ba","Type":"ContainerStarted","Data":"aa90323544f949517e244d96340c461a16b0634f218574db1a8bf68ed6da5d18"} Jul 15 21:06:23.724260 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.723854 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:23.725154 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.725133 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8dv4k" Jul 15 21:06:23.725154 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.725149 2573 scope.go:117] "RemoveContainer" containerID="6d2bddc55aa52137c91247d63caff6e334fc5978dba6e6900e62d68f1a99b39e" Jul 15 21:06:23.726802 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.726775 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" event={"ID":"3373f38d-efe1-4caf-9673-6663cfb5713c","Type":"ContainerStarted","Data":"e562072f3a00771bc26465f3a659349d407f627267d3a8de7be37fa4d861b889"} Jul 15 21:06:23.726901 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.726828 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" event={"ID":"3373f38d-efe1-4caf-9673-6663cfb5713c","Type":"ContainerStarted","Data":"dc18963286c2c5dc2e0e5c9133508668ad4ee497e38f736106bd75c6d6c38025"} Jul 15 21:06:23.726901 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.726855 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:23.761241 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.761195 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" podStartSLOduration=1.761164223 podStartE2EDuration="1.761164223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:23.759185601 +0000 UTC m=+436.849418784" watchObservedRunningTime="2026-07-15 21:06:23.761164223 +0000 UTC m=+436.851397401" Jul 15 21:06:23.805053 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:23.805000 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" podStartSLOduration=1.804984227 podStartE2EDuration="1.804984227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:23.803625393 +0000 UTC m=+436.893858569" watchObservedRunningTime="2026-07-15 21:06:23.804984227 +0000 UTC m=+436.895217405" Jul 15 21:06:34.735843 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:34.735766 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:06:34.735843 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:34.735834 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:34.808671 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:34.808640 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn"] Jul 15 21:06:34.808842 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:34.808817 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" podUID="a5606e5b-d672-4f25-9f88-7910e18bf2ba" containerName="manager" containerID="cri-o://435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878" gracePeriod=10 Jul 15 21:06:35.060677 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.060654 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:35.164466 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.164435 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5606e5b-d672-4f25-9f88-7910e18bf2ba-extensions-socket-volume\") pod \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\" (UID: \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\") " Jul 15 21:06:35.164633 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.164495 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n58t4\" (UniqueName: \"kubernetes.io/projected/a5606e5b-d672-4f25-9f88-7910e18bf2ba-kube-api-access-n58t4\") pod \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\" (UID: \"a5606e5b-d672-4f25-9f88-7910e18bf2ba\") " Jul 15 21:06:35.164860 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.164827 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5606e5b-d672-4f25-9f88-7910e18bf2ba-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a5606e5b-d672-4f25-9f88-7910e18bf2ba" (UID: "a5606e5b-d672-4f25-9f88-7910e18bf2ba"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:06:35.166498 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.166475 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5606e5b-d672-4f25-9f88-7910e18bf2ba-kube-api-access-n58t4" (OuterVolumeSpecName: "kube-api-access-n58t4") pod "a5606e5b-d672-4f25-9f88-7910e18bf2ba" (UID: "a5606e5b-d672-4f25-9f88-7910e18bf2ba"). InnerVolumeSpecName "kube-api-access-n58t4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:06:35.265227 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.265164 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5606e5b-d672-4f25-9f88-7910e18bf2ba-extensions-socket-volume\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:06:35.265227 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.265222 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n58t4\" (UniqueName: \"kubernetes.io/projected/a5606e5b-d672-4f25-9f88-7910e18bf2ba-kube-api-access-n58t4\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:06:35.793501 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.793468 2573 generic.go:356] "Generic (PLEG): container finished" podID="a5606e5b-d672-4f25-9f88-7910e18bf2ba" containerID="435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878" exitCode=0 Jul 15 21:06:35.793868 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.793509 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" Jul 15 21:06:35.793868 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.793535 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" event={"ID":"a5606e5b-d672-4f25-9f88-7910e18bf2ba","Type":"ContainerDied","Data":"435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878"} Jul 15 21:06:35.793868 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.793575 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn" event={"ID":"a5606e5b-d672-4f25-9f88-7910e18bf2ba","Type":"ContainerDied","Data":"aa90323544f949517e244d96340c461a16b0634f218574db1a8bf68ed6da5d18"} Jul 15 21:06:35.793868 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.793595 2573 scope.go:117] "RemoveContainer" containerID="435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878" Jul 15 21:06:35.802507 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.802473 2573 scope.go:117] "RemoveContainer" containerID="435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878" Jul 15 21:06:35.802758 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:06:35.802740 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878\": container with ID starting with 435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878 not found: ID does not exist" containerID="435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878" Jul 15 21:06:35.802802 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.802765 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878"} err="failed to get container status \"435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878\": rpc error: code = NotFound desc = could not find container \"435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878\": container with ID starting with 435bf5ceb74543d2d17ee9788891530adbd4522b2a4f7512d6a9e3c6b8ea5878 not found: ID does not exist" Jul 15 21:06:35.823609 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.823580 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn"] Jul 15 21:06:35.827070 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:35.827045 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mtzdn"] Jul 15 21:06:37.448034 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:37.448003 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5606e5b-d672-4f25-9f88-7910e18bf2ba" path="/var/lib/kubelet/pods/a5606e5b-d672-4f25-9f88-7910e18bf2ba/volumes" Jul 15 21:06:51.187028 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.186986 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp"] Jul 15 21:06:51.187583 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.187554 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5606e5b-d672-4f25-9f88-7910e18bf2ba" containerName="manager" Jul 15 21:06:51.187583 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.187574 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5606e5b-d672-4f25-9f88-7910e18bf2ba" containerName="manager" Jul 15 21:06:51.187775 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.187703 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5606e5b-d672-4f25-9f88-7910e18bf2ba" containerName="manager" Jul 15 21:06:51.190632 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.190611 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.194912 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.194895 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pqpqj\"" Jul 15 21:06:51.218394 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.218361 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp"] Jul 15 21:06:51.300558 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.300525 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.300727 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.300566 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.300727 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.300592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.300727 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.300636 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlbwh\" (UniqueName: \"kubernetes.io/projected/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-kube-api-access-xlbwh\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.300727 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.300679 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.300727 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.300707 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.300889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.300733 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.300889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.300816 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.300889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.300844 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402032 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.401994 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402032 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402034 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402275 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402054 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402275 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlbwh\" (UniqueName: \"kubernetes.io/projected/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-kube-api-access-xlbwh\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402275 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402225 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402393 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402279 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402393 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402324 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402467 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402395 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402467 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402419 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402532 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402477 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402532 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402489 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402808 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402778 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.402931 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.402827 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.403113 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.403095 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.404423 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.404405 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.404879 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.404856 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.415538 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.415509 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlbwh\" (UniqueName: \"kubernetes.io/projected/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-kube-api-access-xlbwh\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.416775 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.416751 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-qf9dp\" (UID: \"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.500444 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.500367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:51.642305 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.642271 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp"] Jul 15 21:06:51.643830 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:06:51.643804 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf0e75e7_6f3a_42b2_9069_6b6f8bd7d4c7.slice/crio-eba8229463b52647d2d49be92dcdcf62e8dafa6edd0d96bbaeb0244f6a525a5e WatchSource:0}: Error finding container eba8229463b52647d2d49be92dcdcf62e8dafa6edd0d96bbaeb0244f6a525a5e: Status 404 returned error can't find the container with id eba8229463b52647d2d49be92dcdcf62e8dafa6edd0d96bbaeb0244f6a525a5e Jul 15 21:06:51.645814 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.645782 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:06:51.645892 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.645846 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:06:51.645892 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.645872 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:06:51.882801 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.882770 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" event={"ID":"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7","Type":"ContainerStarted","Data":"ddd951ec37416a75aade4ec9cb5c5e818ffaa20b556df76e8b454c4b77a52526"} Jul 15 21:06:51.883022 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.882818 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" event={"ID":"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7","Type":"ContainerStarted","Data":"eba8229463b52647d2d49be92dcdcf62e8dafa6edd0d96bbaeb0244f6a525a5e"} Jul 15 21:06:51.920582 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:51.920474 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" podStartSLOduration=0.920453436 podStartE2EDuration="920.453436ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:51.918344306 +0000 UTC m=+465.008577503" watchObservedRunningTime="2026-07-15 21:06:51.920453436 +0000 UTC m=+465.010686615" Jul 15 21:06:52.500863 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:52.500822 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:52.505467 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:52.505442 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:52.887932 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:52.887897 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:52.888811 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:52.888794 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:06:52.948384 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:52.948351 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fxz6c"] Jul 15 21:06:52.951157 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:52.951132 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:06:52.953818 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:52.953796 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 21:06:52.953949 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:52.953897 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4qntx\"" Jul 15 21:06:52.961280 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:52.961255 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fxz6c"] Jul 15 21:06:53.017592 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.017545 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/883c29d6-509e-460b-bf14-9c7b99bbbefd-config-file\") pod \"limitador-limitador-64c94fd64f-fxz6c\" (UID: \"883c29d6-509e-460b-bf14-9c7b99bbbefd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:06:53.017777 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.017689 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjrvj\" (UniqueName: \"kubernetes.io/projected/883c29d6-509e-460b-bf14-9c7b99bbbefd-kube-api-access-gjrvj\") pod \"limitador-limitador-64c94fd64f-fxz6c\" (UID: \"883c29d6-509e-460b-bf14-9c7b99bbbefd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:06:53.045521 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.045491 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fxz6c"] Jul 15 21:06:53.118434 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.118404 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjrvj\" (UniqueName: \"kubernetes.io/projected/883c29d6-509e-460b-bf14-9c7b99bbbefd-kube-api-access-gjrvj\") pod \"limitador-limitador-64c94fd64f-fxz6c\" (UID: \"883c29d6-509e-460b-bf14-9c7b99bbbefd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:06:53.118605 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.118484 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/883c29d6-509e-460b-bf14-9c7b99bbbefd-config-file\") pod \"limitador-limitador-64c94fd64f-fxz6c\" (UID: \"883c29d6-509e-460b-bf14-9c7b99bbbefd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:06:53.119095 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.119076 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/883c29d6-509e-460b-bf14-9c7b99bbbefd-config-file\") pod \"limitador-limitador-64c94fd64f-fxz6c\" (UID: \"883c29d6-509e-460b-bf14-9c7b99bbbefd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:06:53.127152 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.127126 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjrvj\" (UniqueName: \"kubernetes.io/projected/883c29d6-509e-460b-bf14-9c7b99bbbefd-kube-api-access-gjrvj\") pod \"limitador-limitador-64c94fd64f-fxz6c\" (UID: \"883c29d6-509e-460b-bf14-9c7b99bbbefd\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:06:53.263632 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.263597 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:06:53.398855 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.398829 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fxz6c"] Jul 15 21:06:53.400263 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:06:53.400239 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883c29d6_509e_460b_bf14_9c7b99bbbefd.slice/crio-3bca95611babc151cf800c8371f2911614b01e70788ee56716e41192e70e717a WatchSource:0}: Error finding container 3bca95611babc151cf800c8371f2911614b01e70788ee56716e41192e70e717a: Status 404 returned error can't find the container with id 3bca95611babc151cf800c8371f2911614b01e70788ee56716e41192e70e717a Jul 15 21:06:53.894208 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:53.894161 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" event={"ID":"883c29d6-509e-460b-bf14-9c7b99bbbefd","Type":"ContainerStarted","Data":"3bca95611babc151cf800c8371f2911614b01e70788ee56716e41192e70e717a"} Jul 15 21:06:56.915716 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:56.915678 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" event={"ID":"883c29d6-509e-460b-bf14-9c7b99bbbefd","Type":"ContainerStarted","Data":"a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c"} Jul 15 21:06:56.916113 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:56.915875 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:06:56.936188 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:06:56.936119 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" podStartSLOduration=2.146818144 podStartE2EDuration="4.936103994s" totalImagesPullingTime="2.78928585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:06:56.934230547 +0000 UTC m=+470.024463725" watchObservedRunningTime="2026-07-15 21:06:56.936103994 +0000 UTC m=+470.026337173" Jul 15 21:07:07.887354 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:07.887322 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fxz6c"] Jul 15 21:07:07.887803 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:07.887561 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" podUID="883c29d6-509e-460b-bf14-9c7b99bbbefd" containerName="limitador" containerID="cri-o://a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c" gracePeriod=30 Jul 15 21:07:07.888298 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:07.888275 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:07:08.440320 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.440300 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:07:08.559630 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.559601 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/883c29d6-509e-460b-bf14-9c7b99bbbefd-config-file\") pod \"883c29d6-509e-460b-bf14-9c7b99bbbefd\" (UID: \"883c29d6-509e-460b-bf14-9c7b99bbbefd\") " Jul 15 21:07:08.559829 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.559692 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjrvj\" (UniqueName: \"kubernetes.io/projected/883c29d6-509e-460b-bf14-9c7b99bbbefd-kube-api-access-gjrvj\") pod \"883c29d6-509e-460b-bf14-9c7b99bbbefd\" (UID: \"883c29d6-509e-460b-bf14-9c7b99bbbefd\") " Jul 15 21:07:08.560092 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.560055 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/883c29d6-509e-460b-bf14-9c7b99bbbefd-config-file" (OuterVolumeSpecName: "config-file") pod "883c29d6-509e-460b-bf14-9c7b99bbbefd" (UID: "883c29d6-509e-460b-bf14-9c7b99bbbefd"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:07:08.562124 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.562102 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883c29d6-509e-460b-bf14-9c7b99bbbefd-kube-api-access-gjrvj" (OuterVolumeSpecName: "kube-api-access-gjrvj") pod "883c29d6-509e-460b-bf14-9c7b99bbbefd" (UID: "883c29d6-509e-460b-bf14-9c7b99bbbefd"). InnerVolumeSpecName "kube-api-access-gjrvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:07:08.661417 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.661383 2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/883c29d6-509e-460b-bf14-9c7b99bbbefd-config-file\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:07:08.661417 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.661411 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjrvj\" (UniqueName: \"kubernetes.io/projected/883c29d6-509e-460b-bf14-9c7b99bbbefd-kube-api-access-gjrvj\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:07:08.983563 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.983476 2573 generic.go:356] "Generic (PLEG): container finished" podID="883c29d6-509e-460b-bf14-9c7b99bbbefd" containerID="a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c" exitCode=0 Jul 15 21:07:08.983563 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.983517 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" Jul 15 21:07:08.983990 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.983570 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" event={"ID":"883c29d6-509e-460b-bf14-9c7b99bbbefd","Type":"ContainerDied","Data":"a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c"} Jul 15 21:07:08.983990 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.983624 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-fxz6c" event={"ID":"883c29d6-509e-460b-bf14-9c7b99bbbefd","Type":"ContainerDied","Data":"3bca95611babc151cf800c8371f2911614b01e70788ee56716e41192e70e717a"} Jul 15 21:07:08.983990 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.983649 2573 scope.go:117] "RemoveContainer" containerID="a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c" Jul 15 21:07:08.992877 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.992862 2573 scope.go:117] "RemoveContainer" containerID="a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c" Jul 15 21:07:08.993105 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:07:08.993091 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c\": container with ID starting with a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c not found: ID does not exist" containerID="a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c" Jul 15 21:07:08.993150 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:08.993112 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c"} err="failed to get container status \"a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c\": rpc error: code = NotFound desc = could not find container \"a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c\": container with ID starting with a84511b47c4cca68c8f94eb1aeb1798beaeda5f926c784548fdc798fbb61560c not found: ID does not exist" Jul 15 21:07:09.008039 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.008020 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fxz6c"] Jul 15 21:07:09.020069 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.020048 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-fxz6c"] Jul 15 21:07:09.447225 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.447193 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="883c29d6-509e-460b-bf14-9c7b99bbbefd" path="/var/lib/kubelet/pods/883c29d6-509e-460b-bf14-9c7b99bbbefd/volumes" Jul 15 21:07:09.532917 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.532882 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz"] Jul 15 21:07:09.533517 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.533497 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883c29d6-509e-460b-bf14-9c7b99bbbefd" containerName="limitador" Jul 15 21:07:09.533613 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.533520 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="883c29d6-509e-460b-bf14-9c7b99bbbefd" containerName="limitador" Jul 15 21:07:09.533674 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.533626 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="883c29d6-509e-460b-bf14-9c7b99bbbefd" containerName="limitador" Jul 15 21:07:09.538469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.538450 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:09.541538 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.541517 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 21:07:09.542473 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.542448 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-95nb4\"" Jul 15 21:07:09.542473 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.542465 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 15 21:07:09.542634 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.542448 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 15 21:07:09.544432 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.544411 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz"] Jul 15 21:07:09.670853 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.670820 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a74efc64-2f0d-482e-abe9-adec7fa5d89f-data\") pod \"postgres-69fb4594cf-8l4wz\" (UID: \"a74efc64-2f0d-482e-abe9-adec7fa5d89f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:09.670853 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.670855 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2l5z\" (UniqueName: \"kubernetes.io/projected/a74efc64-2f0d-482e-abe9-adec7fa5d89f-kube-api-access-j2l5z\") pod \"postgres-69fb4594cf-8l4wz\" (UID: \"a74efc64-2f0d-482e-abe9-adec7fa5d89f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:09.771661 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.771624 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a74efc64-2f0d-482e-abe9-adec7fa5d89f-data\") pod \"postgres-69fb4594cf-8l4wz\" (UID: \"a74efc64-2f0d-482e-abe9-adec7fa5d89f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:09.771661 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.771664 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2l5z\" (UniqueName: \"kubernetes.io/projected/a74efc64-2f0d-482e-abe9-adec7fa5d89f-kube-api-access-j2l5z\") pod \"postgres-69fb4594cf-8l4wz\" (UID: \"a74efc64-2f0d-482e-abe9-adec7fa5d89f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:09.772000 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.771977 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a74efc64-2f0d-482e-abe9-adec7fa5d89f-data\") pod \"postgres-69fb4594cf-8l4wz\" (UID: \"a74efc64-2f0d-482e-abe9-adec7fa5d89f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:09.781548 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.781523 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2l5z\" (UniqueName: \"kubernetes.io/projected/a74efc64-2f0d-482e-abe9-adec7fa5d89f-kube-api-access-j2l5z\") pod \"postgres-69fb4594cf-8l4wz\" (UID: \"a74efc64-2f0d-482e-abe9-adec7fa5d89f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:09.850210 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.850157 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:09.983286 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:09.983261 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz"] Jul 15 21:07:09.984537 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:07:09.984513 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda74efc64_2f0d_482e_abe9_adec7fa5d89f.slice/crio-6d39e13b6a5ca0436e2c3a601e78eea69da652b1f63251b7a04f94cd06999a83 WatchSource:0}: Error finding container 6d39e13b6a5ca0436e2c3a601e78eea69da652b1f63251b7a04f94cd06999a83: Status 404 returned error can't find the container with id 6d39e13b6a5ca0436e2c3a601e78eea69da652b1f63251b7a04f94cd06999a83 Jul 15 21:07:11.001210 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:11.001154 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" event={"ID":"a74efc64-2f0d-482e-abe9-adec7fa5d89f","Type":"ContainerStarted","Data":"6d39e13b6a5ca0436e2c3a601e78eea69da652b1f63251b7a04f94cd06999a83"} Jul 15 21:07:15.026215 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:15.026165 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" event={"ID":"a74efc64-2f0d-482e-abe9-adec7fa5d89f","Type":"ContainerStarted","Data":"fbab8b95b61568333eedfa8cabf7ac1921beb5dc73c0438fbfefa712ff5893ce"} Jul 15 21:07:15.026589 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:15.026320 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:15.043881 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:15.043825 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" podStartSLOduration=1.3423920599999999 podStartE2EDuration="6.043807652s" totalImagesPullingTime="4.701415592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:15.043512368 +0000 UTC m=+488.133745550" watchObservedRunningTime="2026-07-15 21:07:15.043807652 +0000 UTC m=+488.134040833" Jul 15 21:07:21.062307 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.062273 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-8l4wz" Jul 15 21:07:21.846755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.846719 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-ztjzf"] Jul 15 21:07:21.852982 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.852956 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:21.856433 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.856406 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jlbmc\"" Jul 15 21:07:21.856595 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.856409 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 15 21:07:21.858707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.858684 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-ztjzf"] Jul 15 21:07:21.875578 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.875549 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4e8ea53-d4bb-4329-860f-d31d5c561940-tls-cert\") pod \"authorino-76877994-ztjzf\" (UID: \"a4e8ea53-d4bb-4329-860f-d31d5c561940\") " pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:21.875740 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.875603 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv4r7\" (UniqueName: \"kubernetes.io/projected/a4e8ea53-d4bb-4329-860f-d31d5c561940-kube-api-access-jv4r7\") pod \"authorino-76877994-ztjzf\" (UID: \"a4e8ea53-d4bb-4329-860f-d31d5c561940\") " pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:21.891810 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.891779 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-ztjzf"] Jul 15 21:07:21.892209 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:07:21.892163 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jv4r7 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-ztjzf" podUID="a4e8ea53-d4bb-4329-860f-d31d5c561940" Jul 15 21:07:21.976841 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.976799 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4e8ea53-d4bb-4329-860f-d31d5c561940-tls-cert\") pod \"authorino-76877994-ztjzf\" (UID: \"a4e8ea53-d4bb-4329-860f-d31d5c561940\") " pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:21.977025 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.976850 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv4r7\" (UniqueName: \"kubernetes.io/projected/a4e8ea53-d4bb-4329-860f-d31d5c561940-kube-api-access-jv4r7\") pod \"authorino-76877994-ztjzf\" (UID: \"a4e8ea53-d4bb-4329-860f-d31d5c561940\") " pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:21.979247 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.979218 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4e8ea53-d4bb-4329-860f-d31d5c561940-tls-cert\") pod \"authorino-76877994-ztjzf\" (UID: \"a4e8ea53-d4bb-4329-860f-d31d5c561940\") " pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:21.986522 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:21.986498 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv4r7\" (UniqueName: \"kubernetes.io/projected/a4e8ea53-d4bb-4329-860f-d31d5c561940-kube-api-access-jv4r7\") pod \"authorino-76877994-ztjzf\" (UID: \"a4e8ea53-d4bb-4329-860f-d31d5c561940\") " pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:22.064367 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:22.064338 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:22.072363 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:22.072342 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:22.077241 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:22.077223 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv4r7\" (UniqueName: \"kubernetes.io/projected/a4e8ea53-d4bb-4329-860f-d31d5c561940-kube-api-access-jv4r7\") pod \"a4e8ea53-d4bb-4329-860f-d31d5c561940\" (UID: \"a4e8ea53-d4bb-4329-860f-d31d5c561940\") " Jul 15 21:07:22.077322 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:22.077285 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4e8ea53-d4bb-4329-860f-d31d5c561940-tls-cert\") pod \"a4e8ea53-d4bb-4329-860f-d31d5c561940\" (UID: \"a4e8ea53-d4bb-4329-860f-d31d5c561940\") " Jul 15 21:07:22.079405 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:22.079382 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4e8ea53-d4bb-4329-860f-d31d5c561940-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a4e8ea53-d4bb-4329-860f-d31d5c561940" (UID: "a4e8ea53-d4bb-4329-860f-d31d5c561940"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:07:22.079498 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:22.079423 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4e8ea53-d4bb-4329-860f-d31d5c561940-kube-api-access-jv4r7" (OuterVolumeSpecName: "kube-api-access-jv4r7") pod "a4e8ea53-d4bb-4329-860f-d31d5c561940" (UID: "a4e8ea53-d4bb-4329-860f-d31d5c561940"). InnerVolumeSpecName "kube-api-access-jv4r7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:07:22.177866 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:22.177787 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv4r7\" (UniqueName: \"kubernetes.io/projected/a4e8ea53-d4bb-4329-860f-d31d5c561940-kube-api-access-jv4r7\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:07:22.177866 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:22.177811 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a4e8ea53-d4bb-4329-860f-d31d5c561940-tls-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:07:23.069277 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:23.069245 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-ztjzf" Jul 15 21:07:23.101076 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:23.101046 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-ztjzf"] Jul 15 21:07:23.104908 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:23.104879 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-ztjzf"] Jul 15 21:07:23.447899 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:23.447823 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4e8ea53-d4bb-4329-860f-d31d5c561940" path="/var/lib/kubelet/pods/a4e8ea53-d4bb-4329-860f-d31d5c561940/volumes" Jul 15 21:07:28.204238 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.204197 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f84fb5cd5-78m5b"] Jul 15 21:07:28.228689 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.228657 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f84fb5cd5-78m5b"] Jul 15 21:07:28.228851 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.228760 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:28.231673 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.231642 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 21:07:28.231673 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.231665 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pf5dc\"" Jul 15 21:07:28.231866 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.231714 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 21:07:28.335091 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.335051 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fcf0bbd-88d0-42a0-a944-32c572427d3a-webhook-cert\") pod \"maas-controller-7f84fb5cd5-78m5b\" (UID: \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\") " pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:28.335313 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.335220 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw6ws\" (UniqueName: \"kubernetes.io/projected/9fcf0bbd-88d0-42a0-a944-32c572427d3a-kube-api-access-rw6ws\") pod \"maas-controller-7f84fb5cd5-78m5b\" (UID: \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\") " pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:28.436533 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.436495 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fcf0bbd-88d0-42a0-a944-32c572427d3a-webhook-cert\") pod \"maas-controller-7f84fb5cd5-78m5b\" (UID: \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\") " pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:28.436735 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.436603 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw6ws\" (UniqueName: \"kubernetes.io/projected/9fcf0bbd-88d0-42a0-a944-32c572427d3a-kube-api-access-rw6ws\") pod \"maas-controller-7f84fb5cd5-78m5b\" (UID: \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\") " pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:28.436735 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:07:28.436663 2573 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 15 21:07:28.436863 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:07:28.436738 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fcf0bbd-88d0-42a0-a944-32c572427d3a-webhook-cert podName:9fcf0bbd-88d0-42a0-a944-32c572427d3a nodeName:}" failed. No retries permitted until 2026-07-15 21:07:28.936721837 +0000 UTC m=+502.026954993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9fcf0bbd-88d0-42a0-a944-32c572427d3a-webhook-cert") pod "maas-controller-7f84fb5cd5-78m5b" (UID: "9fcf0bbd-88d0-42a0-a944-32c572427d3a") : secret "maas-controller-webhook-cert" not found Jul 15 21:07:28.445968 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.445943 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw6ws\" (UniqueName: \"kubernetes.io/projected/9fcf0bbd-88d0-42a0-a944-32c572427d3a-kube-api-access-rw6ws\") pod \"maas-controller-7f84fb5cd5-78m5b\" (UID: \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\") " pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:28.940540 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.940505 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fcf0bbd-88d0-42a0-a944-32c572427d3a-webhook-cert\") pod \"maas-controller-7f84fb5cd5-78m5b\" (UID: \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\") " pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:28.942775 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:28.942754 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fcf0bbd-88d0-42a0-a944-32c572427d3a-webhook-cert\") pod \"maas-controller-7f84fb5cd5-78m5b\" (UID: \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\") " pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:29.139886 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:29.139845 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:29.297021 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:29.296971 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f84fb5cd5-78m5b"] Jul 15 21:07:29.299638 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:07:29.299612 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fcf0bbd_88d0_42a0_a944_32c572427d3a.slice/crio-daeecc17aac47973c0a691a75b75bb69705c3dd3cddecd8cf8d2d692f4710312 WatchSource:0}: Error finding container daeecc17aac47973c0a691a75b75bb69705c3dd3cddecd8cf8d2d692f4710312: Status 404 returned error can't find the container with id daeecc17aac47973c0a691a75b75bb69705c3dd3cddecd8cf8d2d692f4710312 Jul 15 21:07:30.114295 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:30.114225 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" event={"ID":"9fcf0bbd-88d0-42a0-a944-32c572427d3a","Type":"ContainerStarted","Data":"daeecc17aac47973c0a691a75b75bb69705c3dd3cddecd8cf8d2d692f4710312"} Jul 15 21:07:32.127609 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:32.127517 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" event={"ID":"9fcf0bbd-88d0-42a0-a944-32c572427d3a","Type":"ContainerStarted","Data":"d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab"} Jul 15 21:07:32.127609 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:32.127587 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:32.148462 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:32.148412 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" podStartSLOduration=1.5878712130000001 podStartE2EDuration="4.148397944s" totalImagesPullingTime="2.560526731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:07:32.14744581 +0000 UTC m=+505.237678988" watchObservedRunningTime="2026-07-15 21:07:32.148397944 +0000 UTC m=+505.238631122" Jul 15 21:07:43.138285 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.138251 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:43.533632 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.533598 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f84fb5cd5-78m5b"] Jul 15 21:07:43.533830 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.533777 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" podUID="9fcf0bbd-88d0-42a0-a944-32c572427d3a" containerName="manager" containerID="cri-o://d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab" gracePeriod=30 Jul 15 21:07:43.780624 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.780601 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:43.886889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.886810 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fcf0bbd-88d0-42a0-a944-32c572427d3a-webhook-cert\") pod \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\" (UID: \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\") " Jul 15 21:07:43.886889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.886864 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw6ws\" (UniqueName: \"kubernetes.io/projected/9fcf0bbd-88d0-42a0-a944-32c572427d3a-kube-api-access-rw6ws\") pod \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\" (UID: \"9fcf0bbd-88d0-42a0-a944-32c572427d3a\") " Jul 15 21:07:43.889122 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.889088 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fcf0bbd-88d0-42a0-a944-32c572427d3a-kube-api-access-rw6ws" (OuterVolumeSpecName: "kube-api-access-rw6ws") pod "9fcf0bbd-88d0-42a0-a944-32c572427d3a" (UID: "9fcf0bbd-88d0-42a0-a944-32c572427d3a"). InnerVolumeSpecName "kube-api-access-rw6ws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:07:43.889267 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.889115 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fcf0bbd-88d0-42a0-a944-32c572427d3a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9fcf0bbd-88d0-42a0-a944-32c572427d3a" (UID: "9fcf0bbd-88d0-42a0-a944-32c572427d3a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:07:43.987743 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.987705 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9fcf0bbd-88d0-42a0-a944-32c572427d3a-webhook-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:07:43.987743 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:43.987736 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw6ws\" (UniqueName: \"kubernetes.io/projected/9fcf0bbd-88d0-42a0-a944-32c572427d3a-kube-api-access-rw6ws\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:07:44.200712 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:44.200623 2573 generic.go:356] "Generic (PLEG): container finished" podID="9fcf0bbd-88d0-42a0-a944-32c572427d3a" containerID="d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab" exitCode=0 Jul 15 21:07:44.200712 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:44.200670 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" Jul 15 21:07:44.201191 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:44.200710 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" event={"ID":"9fcf0bbd-88d0-42a0-a944-32c572427d3a","Type":"ContainerDied","Data":"d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab"} Jul 15 21:07:44.201191 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:44.200769 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f84fb5cd5-78m5b" event={"ID":"9fcf0bbd-88d0-42a0-a944-32c572427d3a","Type":"ContainerDied","Data":"daeecc17aac47973c0a691a75b75bb69705c3dd3cddecd8cf8d2d692f4710312"} Jul 15 21:07:44.201191 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:44.200791 2573 scope.go:117] "RemoveContainer" containerID="d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab" Jul 15 21:07:44.210009 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:44.209990 2573 scope.go:117] "RemoveContainer" containerID="d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab" Jul 15 21:07:44.210299 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:07:44.210281 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab\": container with ID starting with d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab not found: ID does not exist" containerID="d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab" Jul 15 21:07:44.210369 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:44.210308 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab"} err="failed to get container status \"d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab\": rpc error: code = NotFound desc = could not find container \"d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab\": container with ID starting with d0cbe7eb7a48d8eec589d03aa3d15baab0c3b5d92e094f61f5c9ac4a8c4bc6ab not found: ID does not exist" Jul 15 21:07:44.223467 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:44.223445 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f84fb5cd5-78m5b"] Jul 15 21:07:44.227361 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:44.227341 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f84fb5cd5-78m5b"] Jul 15 21:07:45.447797 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:07:45.447755 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fcf0bbd-88d0-42a0-a944-32c572427d3a" path="/var/lib/kubelet/pods/9fcf0bbd-88d0-42a0-a944-32c572427d3a/volumes" Jul 15 21:08:04.704962 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.704920 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp"] Jul 15 21:08:04.705463 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.705444 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fcf0bbd-88d0-42a0-a944-32c572427d3a" containerName="manager" Jul 15 21:08:04.705508 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.705465 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fcf0bbd-88d0-42a0-a944-32c572427d3a" containerName="manager" Jul 15 21:08:04.705543 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.705538 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fcf0bbd-88d0-42a0-a944-32c572427d3a" containerName="manager" Jul 15 21:08:04.708351 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.708332 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:04.711228 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.711210 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wdxsw\"" Jul 15 21:08:04.711333 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.711239 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 15 21:08:04.720968 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.720948 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp"] Jul 15 21:08:04.772396 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.772367 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d429723-8df7-4cf2-8794-1b75262b7121-openshift-service-ca\") pod \"maas-api-7f9664fdf-lrldp\" (UID: \"5d429723-8df7-4cf2-8794-1b75262b7121\") " pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:04.772581 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.772403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsjw9\" (UniqueName: \"kubernetes.io/projected/5d429723-8df7-4cf2-8794-1b75262b7121-kube-api-access-xsjw9\") pod \"maas-api-7f9664fdf-lrldp\" (UID: \"5d429723-8df7-4cf2-8794-1b75262b7121\") " pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:04.772581 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.772535 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d429723-8df7-4cf2-8794-1b75262b7121-maas-api-tls\") pod \"maas-api-7f9664fdf-lrldp\" (UID: \"5d429723-8df7-4cf2-8794-1b75262b7121\") " pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:04.873860 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.873827 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d429723-8df7-4cf2-8794-1b75262b7121-maas-api-tls\") pod \"maas-api-7f9664fdf-lrldp\" (UID: \"5d429723-8df7-4cf2-8794-1b75262b7121\") " pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:04.874043 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.873888 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d429723-8df7-4cf2-8794-1b75262b7121-openshift-service-ca\") pod \"maas-api-7f9664fdf-lrldp\" (UID: \"5d429723-8df7-4cf2-8794-1b75262b7121\") " pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:04.874043 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.873997 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsjw9\" (UniqueName: \"kubernetes.io/projected/5d429723-8df7-4cf2-8794-1b75262b7121-kube-api-access-xsjw9\") pod \"maas-api-7f9664fdf-lrldp\" (UID: \"5d429723-8df7-4cf2-8794-1b75262b7121\") " pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:04.874476 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.874457 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d429723-8df7-4cf2-8794-1b75262b7121-openshift-service-ca\") pod \"maas-api-7f9664fdf-lrldp\" (UID: \"5d429723-8df7-4cf2-8794-1b75262b7121\") " pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:04.876282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.876261 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d429723-8df7-4cf2-8794-1b75262b7121-maas-api-tls\") pod \"maas-api-7f9664fdf-lrldp\" (UID: \"5d429723-8df7-4cf2-8794-1b75262b7121\") " pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:04.892078 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:04.892052 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsjw9\" (UniqueName: \"kubernetes.io/projected/5d429723-8df7-4cf2-8794-1b75262b7121-kube-api-access-xsjw9\") pod \"maas-api-7f9664fdf-lrldp\" (UID: \"5d429723-8df7-4cf2-8794-1b75262b7121\") " pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:05.019933 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.019895 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:05.164538 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.164510 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp"] Jul 15 21:08:05.166527 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:08:05.166495 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d429723_8df7_4cf2_8794_1b75262b7121.slice/crio-47a2acc68d65e94118109ab957f1776a92ab18682ead5ec091a69a997c703d1e WatchSource:0}: Error finding container 47a2acc68d65e94118109ab957f1776a92ab18682ead5ec091a69a997c703d1e: Status 404 returned error can't find the container with id 47a2acc68d65e94118109ab957f1776a92ab18682ead5ec091a69a997c703d1e Jul 15 21:08:05.322888 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.322816 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" event={"ID":"5d429723-8df7-4cf2-8794-1b75262b7121","Type":"ContainerStarted","Data":"47a2acc68d65e94118109ab957f1776a92ab18682ead5ec091a69a997c703d1e"} Jul 15 21:08:05.352126 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.352091 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-l6hpz"] Jul 15 21:08:05.390484 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.390454 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-l6hpz"] Jul 15 21:08:05.390644 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.390560 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:05.393574 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.393553 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 21:08:05.393677 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.393580 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sjj6d\"" Jul 15 21:08:05.478551 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.478512 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cf3a567-8ba9-41a3-8b71-9296626ccf35-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-l6hpz\" (UID: \"8cf3a567-8ba9-41a3-8b71-9296626ccf35\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:05.478551 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.478555 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gvfk\" (UniqueName: \"kubernetes.io/projected/8cf3a567-8ba9-41a3-8b71-9296626ccf35-kube-api-access-5gvfk\") pod \"payload-processing-f9b8d9b9b-l6hpz\" (UID: \"8cf3a567-8ba9-41a3-8b71-9296626ccf35\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:05.579287 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.579212 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cf3a567-8ba9-41a3-8b71-9296626ccf35-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-l6hpz\" (UID: \"8cf3a567-8ba9-41a3-8b71-9296626ccf35\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:05.579287 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.579257 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gvfk\" (UniqueName: \"kubernetes.io/projected/8cf3a567-8ba9-41a3-8b71-9296626ccf35-kube-api-access-5gvfk\") pod \"payload-processing-f9b8d9b9b-l6hpz\" (UID: \"8cf3a567-8ba9-41a3-8b71-9296626ccf35\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:05.579891 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.579869 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8cf3a567-8ba9-41a3-8b71-9296626ccf35-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-l6hpz\" (UID: \"8cf3a567-8ba9-41a3-8b71-9296626ccf35\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:05.588886 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.588860 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gvfk\" (UniqueName: \"kubernetes.io/projected/8cf3a567-8ba9-41a3-8b71-9296626ccf35-kube-api-access-5gvfk\") pod \"payload-processing-f9b8d9b9b-l6hpz\" (UID: \"8cf3a567-8ba9-41a3-8b71-9296626ccf35\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:05.701424 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.701390 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:05.850108 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:08:05.850068 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf3a567_8ba9_41a3_8b71_9296626ccf35.slice/crio-d612718b14bfd1aea078fe1237a58c242b7777078933db191dea3fe04fc47c0e WatchSource:0}: Error finding container d612718b14bfd1aea078fe1237a58c242b7777078933db191dea3fe04fc47c0e: Status 404 returned error can't find the container with id d612718b14bfd1aea078fe1237a58c242b7777078933db191dea3fe04fc47c0e Jul 15 21:08:05.854799 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:05.854772 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-l6hpz"] Jul 15 21:08:06.359977 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:06.359934 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" event={"ID":"8cf3a567-8ba9-41a3-8b71-9296626ccf35","Type":"ContainerStarted","Data":"d612718b14bfd1aea078fe1237a58c242b7777078933db191dea3fe04fc47c0e"} Jul 15 21:08:08.374400 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:08.374360 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" event={"ID":"5d429723-8df7-4cf2-8794-1b75262b7121","Type":"ContainerStarted","Data":"36e2d9977822eeebdb0de60cca101f8872b332b07394cae7016fe6c376459fd0"} Jul 15 21:08:08.374803 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:08.374498 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:08.376136 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:08.376106 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" event={"ID":"8cf3a567-8ba9-41a3-8b71-9296626ccf35","Type":"ContainerStarted","Data":"b62c6227a8c6409a625e74350ae9f187b0c8fe15d04733180aa5349a40b7af79"} Jul 15 21:08:08.376314 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:08.376159 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:08.395774 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:08.395719 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" podStartSLOduration=1.752509506 podStartE2EDuration="4.395700853s" totalImagesPullingTime="2.643191347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:08.394191219 +0000 UTC m=+541.484424390" watchObservedRunningTime="2026-07-15 21:08:08.395700853 +0000 UTC m=+541.485934030" Jul 15 21:08:08.412102 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:08.412053 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" podStartSLOduration=1.453585529 podStartE2EDuration="3.412036071s" totalImagesPullingTime="1.958450542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:08.411463563 +0000 UTC m=+541.501696743" watchObservedRunningTime="2026-07-15 21:08:08.412036071 +0000 UTC m=+541.502269254" Jul 15 21:08:13.414664 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:13.414578 2573 generic.go:356] "Generic (PLEG): container finished" podID="cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7" containerID="ddd951ec37416a75aade4ec9cb5c5e818ffaa20b556df76e8b454c4b77a52526" exitCode=0 Jul 15 21:08:13.414664 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:13.414651 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" event={"ID":"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7","Type":"ContainerDied","Data":"ddd951ec37416a75aade4ec9cb5c5e818ffaa20b556df76e8b454c4b77a52526"} Jul 15 21:08:13.415157 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:13.415047 2573 scope.go:117] "RemoveContainer" containerID="ddd951ec37416a75aade4ec9cb5c5e818ffaa20b556df76e8b454c4b77a52526" Jul 15 21:08:13.416008 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:13.415983 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:08:13.416063 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:13.416051 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:08:13.416102 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:13.416086 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:08:14.387763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:14.387735 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f9664fdf-lrldp" Jul 15 21:08:14.424437 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:14.424391 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" event={"ID":"cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7","Type":"ContainerStarted","Data":"b4a18e6c36fd480b709371faeb12fc12496c87e9d629929aed9511b4cfe69f5b"} Jul 15 21:08:14.500750 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:14.500719 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:08:14.500933 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:14.500911 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qf9dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 21:08:14.500979 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:14.500961 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" podUID="cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 21:08:15.500773 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:15.500741 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qf9dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 21:08:15.501166 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:15.500795 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" podUID="cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 21:08:16.501631 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:16.501593 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qf9dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 21:08:16.502016 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:16.501647 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" podUID="cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 21:08:17.501387 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:17.501349 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-qf9dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 21:08:17.501561 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:17.501400 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" podUID="cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 21:08:18.505189 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:18.505142 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:08:18.505535 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:18.505494 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:08:18.506107 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:18.506093 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-qf9dp" Jul 15 21:08:19.383965 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:19.383937 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-l6hpz" Jul 15 21:08:48.299514 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.299475 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw"] Jul 15 21:08:48.303105 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.303083 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.305993 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.305974 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 15 21:08:48.307025 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.307007 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 21:08:48.307282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.307266 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 21:08:48.307355 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.307314 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vqths\"" Jul 15 21:08:48.314209 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.314164 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw"] Jul 15 21:08:48.376025 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.375985 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.376237 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.376049 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.376237 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.376082 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40f9af7b-827b-40ab-97f4-0ca2f59f8747-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.376237 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.376110 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll6hw\" (UniqueName: \"kubernetes.io/projected/40f9af7b-827b-40ab-97f4-0ca2f59f8747-kube-api-access-ll6hw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.376237 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.376143 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.376454 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.376266 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.476824 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.476791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.477003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.476837 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.477003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.476861 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40f9af7b-827b-40ab-97f4-0ca2f59f8747-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.477003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.476886 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll6hw\" (UniqueName: \"kubernetes.io/projected/40f9af7b-827b-40ab-97f4-0ca2f59f8747-kube-api-access-ll6hw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.477003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.476913 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.477266 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.477019 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.477355 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.477330 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.477413 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.477377 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.477642 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.477618 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.479020 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.479003 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40f9af7b-827b-40ab-97f4-0ca2f59f8747-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.479391 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.479374 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40f9af7b-827b-40ab-97f4-0ca2f59f8747-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.486649 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.486624 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll6hw\" (UniqueName: \"kubernetes.io/projected/40f9af7b-827b-40ab-97f4-0ca2f59f8747-kube-api-access-ll6hw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-74gzw\" (UID: \"40f9af7b-827b-40ab-97f4-0ca2f59f8747\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.615015 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.614916 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:48.753431 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:48.753402 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw"] Jul 15 21:08:48.754778 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:08:48.754749 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f9af7b_827b_40ab_97f4_0ca2f59f8747.slice/crio-b1368eb242986a56ca2254a27220e09fed4a1c44470d8f4075fa33dca751fed2 WatchSource:0}: Error finding container b1368eb242986a56ca2254a27220e09fed4a1c44470d8f4075fa33dca751fed2: Status 404 returned error can't find the container with id b1368eb242986a56ca2254a27220e09fed4a1c44470d8f4075fa33dca751fed2 Jul 15 21:08:49.638644 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:49.638608 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" event={"ID":"40f9af7b-827b-40ab-97f4-0ca2f59f8747","Type":"ContainerStarted","Data":"b1368eb242986a56ca2254a27220e09fed4a1c44470d8f4075fa33dca751fed2"} Jul 15 21:08:50.645844 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:50.645810 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" event={"ID":"40f9af7b-827b-40ab-97f4-0ca2f59f8747","Type":"ContainerStarted","Data":"ae18a43bb51cd84beab6e9b94d04922ec7d4e4fe0acfeb458df8d65fbe448eb4"} Jul 15 21:08:50.666433 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:50.666379 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" podStartSLOduration=1.63404587 podStartE2EDuration="2.666363209s" totalImagesPullingTime="1.032317339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:08:50.664612157 +0000 UTC m=+583.754845354" watchObservedRunningTime="2026-07-15 21:08:50.666363209 +0000 UTC m=+583.756596387" Jul 15 21:08:58.615443 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:58.615399 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:58.615907 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:58.615500 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:58.628430 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:58.628408 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:08:58.702607 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:08:58.702581 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-74gzw" Jul 15 21:09:07.352805 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:09:07.352771 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:09:07.353364 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:09:07.352872 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:09:07.363355 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:09:07.363333 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:09:07.363355 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:09:07.363342 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:11:27.204289 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.204252 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59d5c68977-fhdzm"] Jul 15 21:11:27.207473 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.207457 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:11:27.212040 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.212019 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pf5dc\"" Jul 15 21:11:27.212200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.212057 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 15 21:11:27.212200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.212116 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 15 21:11:27.224982 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.224955 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59d5c68977-fhdzm"] Jul 15 21:11:27.309766 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.309738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8ds2\" (UniqueName: \"kubernetes.io/projected/c3ca1567-0a3b-49e8-b166-c368e2d87553-kube-api-access-p8ds2\") pod \"maas-controller-59d5c68977-fhdzm\" (UID: \"c3ca1567-0a3b-49e8-b166-c368e2d87553\") " pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:11:27.309931 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.309776 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3ca1567-0a3b-49e8-b166-c368e2d87553-webhook-cert\") pod \"maas-controller-59d5c68977-fhdzm\" (UID: \"c3ca1567-0a3b-49e8-b166-c368e2d87553\") " pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:11:27.410553 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.410525 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3ca1567-0a3b-49e8-b166-c368e2d87553-webhook-cert\") pod \"maas-controller-59d5c68977-fhdzm\" (UID: \"c3ca1567-0a3b-49e8-b166-c368e2d87553\") " pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:11:27.410733 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.410678 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8ds2\" (UniqueName: \"kubernetes.io/projected/c3ca1567-0a3b-49e8-b166-c368e2d87553-kube-api-access-p8ds2\") pod \"maas-controller-59d5c68977-fhdzm\" (UID: \"c3ca1567-0a3b-49e8-b166-c368e2d87553\") " pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:11:27.413150 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.413126 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3ca1567-0a3b-49e8-b166-c368e2d87553-webhook-cert\") pod \"maas-controller-59d5c68977-fhdzm\" (UID: \"c3ca1567-0a3b-49e8-b166-c368e2d87553\") " pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:11:27.420501 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.420477 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8ds2\" (UniqueName: \"kubernetes.io/projected/c3ca1567-0a3b-49e8-b166-c368e2d87553-kube-api-access-p8ds2\") pod \"maas-controller-59d5c68977-fhdzm\" (UID: \"c3ca1567-0a3b-49e8-b166-c368e2d87553\") " pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:11:27.517978 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.517952 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:11:27.666528 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.666507 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59d5c68977-fhdzm"] Jul 15 21:11:27.667706 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:11:27.667681 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ca1567_0a3b_49e8_b166_c368e2d87553.slice/crio-c4692184c986cf05a573b21bab47626138e18d8b13f951eca80c45078d6865ac WatchSource:0}: Error finding container c4692184c986cf05a573b21bab47626138e18d8b13f951eca80c45078d6865ac: Status 404 returned error can't find the container with id c4692184c986cf05a573b21bab47626138e18d8b13f951eca80c45078d6865ac Jul 15 21:11:27.669125 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:27.669108 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:11:28.530645 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:28.530610 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59d5c68977-fhdzm" event={"ID":"c3ca1567-0a3b-49e8-b166-c368e2d87553","Type":"ContainerStarted","Data":"0bdf5cc379697daf6b4ab0061dabbc32288654f2f925a552f20018f3d040506d"} Jul 15 21:11:28.531024 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:28.530677 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59d5c68977-fhdzm" event={"ID":"c3ca1567-0a3b-49e8-b166-c368e2d87553","Type":"ContainerStarted","Data":"c4692184c986cf05a573b21bab47626138e18d8b13f951eca80c45078d6865ac"} Jul 15 21:11:28.531024 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:28.530723 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:11:28.551716 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:28.551672 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59d5c68977-fhdzm" podStartSLOduration=1.071129494 podStartE2EDuration="1.551657995s" totalImagesPullingTime="480.528501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:11:28.550445176 +0000 UTC m=+741.640678354" watchObservedRunningTime="2026-07-15 21:11:28.551657995 +0000 UTC m=+741.641891173" Jul 15 21:11:39.538258 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:11:39.538229 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59d5c68977-fhdzm" Jul 15 21:14:07.398332 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:14:07.398301 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:14:07.399651 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:14:07.399630 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:14:07.402477 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:14:07.402448 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:14:07.403872 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:14:07.403851 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:15:00.145705 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:00.145673 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9"] Jul 15 21:15:00.148609 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:00.148591 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9" Jul 15 21:15:00.151613 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:00.151592 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kpm6w\"" Jul 15 21:15:00.158299 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:00.158274 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9" Jul 15 21:15:00.172227 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:00.171469 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9"] Jul 15 21:15:00.301436 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:00.301410 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9"] Jul 15 21:15:00.302976 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:15:00.302950 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23ffc78_e3d8_4ace_9603_eaa5290752c0.slice/crio-220e1f0c249a8169d37ae8350b8c869881f296b870108ff7600bce7b897af18a WatchSource:0}: Error finding container 220e1f0c249a8169d37ae8350b8c869881f296b870108ff7600bce7b897af18a: Status 404 returned error can't find the container with id 220e1f0c249a8169d37ae8350b8c869881f296b870108ff7600bce7b897af18a Jul 15 21:15:00.731049 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:00.731011 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9" event={"ID":"c23ffc78-e3d8-4ace-9603-eaa5290752c0","Type":"ContainerStarted","Data":"220e1f0c249a8169d37ae8350b8c869881f296b870108ff7600bce7b897af18a"} Jul 15 21:15:03.750612 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:03.750526 2573 generic.go:356] "Generic (PLEG): container finished" podID="c23ffc78-e3d8-4ace-9603-eaa5290752c0" containerID="ef57f29eb03afae121359b83ee4d8f771844653cf9079df6b3178c17bbe13c93" exitCode=0 Jul 15 21:15:03.750612 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:03.750579 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9" event={"ID":"c23ffc78-e3d8-4ace-9603-eaa5290752c0","Type":"ContainerDied","Data":"ef57f29eb03afae121359b83ee4d8f771844653cf9079df6b3178c17bbe13c93"} Jul 15 21:15:04.895696 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:04.895670 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9" Jul 15 21:15:05.771414 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:05.771386 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9" Jul 15 21:15:05.771574 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:05.771416 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735835-blmh9" event={"ID":"c23ffc78-e3d8-4ace-9603-eaa5290752c0","Type":"ContainerDied","Data":"220e1f0c249a8169d37ae8350b8c869881f296b870108ff7600bce7b897af18a"} Jul 15 21:15:05.771574 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:15:05.771463 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="220e1f0c249a8169d37ae8350b8c869881f296b870108ff7600bce7b897af18a" Jul 15 21:16:15.810453 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:15.810415 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm"] Jul 15 21:16:15.810937 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:15.810602 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" podUID="3373f38d-efe1-4caf-9673-6663cfb5713c" containerName="manager" containerID="cri-o://e562072f3a00771bc26465f3a659349d407f627267d3a8de7be37fa4d861b889" gracePeriod=10 Jul 15 21:16:17.178825 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:17.178794 2573 generic.go:356] "Generic (PLEG): container finished" podID="3373f38d-efe1-4caf-9673-6663cfb5713c" containerID="e562072f3a00771bc26465f3a659349d407f627267d3a8de7be37fa4d861b889" exitCode=0 Jul 15 21:16:17.179193 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:17.178872 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" event={"ID":"3373f38d-efe1-4caf-9673-6663cfb5713c","Type":"ContainerDied","Data":"e562072f3a00771bc26465f3a659349d407f627267d3a8de7be37fa4d861b889"} Jul 15 21:16:17.276128 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:17.276102 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:16:17.391066 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:17.390979 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3373f38d-efe1-4caf-9673-6663cfb5713c-extensions-socket-volume\") pod \"3373f38d-efe1-4caf-9673-6663cfb5713c\" (UID: \"3373f38d-efe1-4caf-9673-6663cfb5713c\") " Jul 15 21:16:17.391245 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:17.391071 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfzn7\" (UniqueName: \"kubernetes.io/projected/3373f38d-efe1-4caf-9673-6663cfb5713c-kube-api-access-jfzn7\") pod \"3373f38d-efe1-4caf-9673-6663cfb5713c\" (UID: \"3373f38d-efe1-4caf-9673-6663cfb5713c\") " Jul 15 21:16:17.391355 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:17.391332 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3373f38d-efe1-4caf-9673-6663cfb5713c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3373f38d-efe1-4caf-9673-6663cfb5713c" (UID: "3373f38d-efe1-4caf-9673-6663cfb5713c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:16:17.392950 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:17.392925 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3373f38d-efe1-4caf-9673-6663cfb5713c-kube-api-access-jfzn7" (OuterVolumeSpecName: "kube-api-access-jfzn7") pod "3373f38d-efe1-4caf-9673-6663cfb5713c" (UID: "3373f38d-efe1-4caf-9673-6663cfb5713c"). InnerVolumeSpecName "kube-api-access-jfzn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:16:17.491969 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:17.491935 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfzn7\" (UniqueName: \"kubernetes.io/projected/3373f38d-efe1-4caf-9673-6663cfb5713c-kube-api-access-jfzn7\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:16:17.491969 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:17.491968 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3373f38d-efe1-4caf-9673-6663cfb5713c-extensions-socket-volume\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:16:18.188281 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:18.188249 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" Jul 15 21:16:18.188766 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:18.188276 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm" event={"ID":"3373f38d-efe1-4caf-9673-6663cfb5713c","Type":"ContainerDied","Data":"dc18963286c2c5dc2e0e5c9133508668ad4ee497e38f736106bd75c6d6c38025"} Jul 15 21:16:18.188766 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:18.188343 2573 scope.go:117] "RemoveContainer" containerID="e562072f3a00771bc26465f3a659349d407f627267d3a8de7be37fa4d861b889" Jul 15 21:16:18.210778 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:18.210751 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm"] Jul 15 21:16:18.215290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:18.215267 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nm2dm"] Jul 15 21:16:19.447325 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:16:19.447293 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3373f38d-efe1-4caf-9673-6663cfb5713c" path="/var/lib/kubelet/pods/3373f38d-efe1-4caf-9673-6663cfb5713c/volumes" Jul 15 21:17:17.031691 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.031657 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh"] Jul 15 21:17:17.032071 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.032058 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c23ffc78-e3d8-4ace-9603-eaa5290752c0" containerName="cleanup" Jul 15 21:17:17.032071 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.032068 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c23ffc78-e3d8-4ace-9603-eaa5290752c0" containerName="cleanup" Jul 15 21:17:17.032156 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.032077 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3373f38d-efe1-4caf-9673-6663cfb5713c" containerName="manager" Jul 15 21:17:17.032156 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.032083 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3373f38d-efe1-4caf-9673-6663cfb5713c" containerName="manager" Jul 15 21:17:17.032156 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.032148 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3373f38d-efe1-4caf-9673-6663cfb5713c" containerName="manager" Jul 15 21:17:17.032156 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.032154 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c23ffc78-e3d8-4ace-9603-eaa5290752c0" containerName="cleanup" Jul 15 21:17:17.035379 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.035354 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:17:17.038962 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.038940 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pcsdv\"" Jul 15 21:17:17.055064 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.055043 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh"] Jul 15 21:17:17.134095 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.134070 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/64f52d57-004a-4fee-a8f1-36e939ccab01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jwdsh\" (UID: \"64f52d57-004a-4fee-a8f1-36e939ccab01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:17:17.134241 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.134100 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drhp8\" (UniqueName: \"kubernetes.io/projected/64f52d57-004a-4fee-a8f1-36e939ccab01-kube-api-access-drhp8\") pod \"kuadrant-operator-controller-manager-86878c769f-jwdsh\" (UID: \"64f52d57-004a-4fee-a8f1-36e939ccab01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:17:17.235541 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.235509 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/64f52d57-004a-4fee-a8f1-36e939ccab01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jwdsh\" (UID: \"64f52d57-004a-4fee-a8f1-36e939ccab01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:17:17.235541 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.235539 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drhp8\" (UniqueName: \"kubernetes.io/projected/64f52d57-004a-4fee-a8f1-36e939ccab01-kube-api-access-drhp8\") pod \"kuadrant-operator-controller-manager-86878c769f-jwdsh\" (UID: \"64f52d57-004a-4fee-a8f1-36e939ccab01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:17:17.235835 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.235812 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/64f52d57-004a-4fee-a8f1-36e939ccab01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jwdsh\" (UID: \"64f52d57-004a-4fee-a8f1-36e939ccab01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:17:17.247055 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.247033 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drhp8\" (UniqueName: \"kubernetes.io/projected/64f52d57-004a-4fee-a8f1-36e939ccab01-kube-api-access-drhp8\") pod \"kuadrant-operator-controller-manager-86878c769f-jwdsh\" (UID: \"64f52d57-004a-4fee-a8f1-36e939ccab01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:17:17.345511 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.345427 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:17:17.494651 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.494625 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh"] Jul 15 21:17:17.496846 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:17:17.496823 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64f52d57_004a_4fee_a8f1_36e939ccab01.slice/crio-9aaad16cc5779393a9b1ccc351ab8330958a127712f78a9bfe769c05cee7f8c0 WatchSource:0}: Error finding container 9aaad16cc5779393a9b1ccc351ab8330958a127712f78a9bfe769c05cee7f8c0: Status 404 returned error can't find the container with id 9aaad16cc5779393a9b1ccc351ab8330958a127712f78a9bfe769c05cee7f8c0 Jul 15 21:17:17.515499 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:17.515471 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" event={"ID":"64f52d57-004a-4fee-a8f1-36e939ccab01","Type":"ContainerStarted","Data":"9aaad16cc5779393a9b1ccc351ab8330958a127712f78a9bfe769c05cee7f8c0"} Jul 15 21:17:18.522492 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:18.522457 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" event={"ID":"64f52d57-004a-4fee-a8f1-36e939ccab01","Type":"ContainerStarted","Data":"9fb96b9cd5ffa166c0aea33aa3e80aa34658e4a632e3175e73fa69707a53970e"} Jul 15 21:17:18.522931 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:18.522570 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:17:18.549267 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:18.549227 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" podStartSLOduration=2.549213719 podStartE2EDuration="2.549213719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:17:18.546965647 +0000 UTC m=+1091.637198825" watchObservedRunningTime="2026-07-15 21:17:18.549213719 +0000 UTC m=+1091.639446898" Jul 15 21:17:29.530321 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:17:29.530290 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jwdsh" Jul 15 21:19:07.448685 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:19:07.448659 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:19:07.451577 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:19:07.449950 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:19:07.452829 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:19:07.452807 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:19:07.453770 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:19:07.453747 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:24:03.660949 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.660915 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq"] Jul 15 21:24:03.664535 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.664518 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.668892 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.668871 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0b8f8143-openshift-default-dockercfg-hqb87\"" Jul 15 21:24:03.709717 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.709687 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq"] Jul 15 21:24:03.848474 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.848440 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-istio-token\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.848663 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.848562 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-data\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.848663 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.848591 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cgbr\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-kube-api-access-9cgbr\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.848763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.848666 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-socket\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.848763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.848688 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83da9ca-9094-4fd8-8069-b91895d20613-istio-podinfo\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.848763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.848709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-certs\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.848763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.848756 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83da9ca-9094-4fd8-8069-b91895d20613-istiod-ca-cert\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.849265 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.848781 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-envoy\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.849265 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.848819 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-credential-socket\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.949996 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.949913 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83da9ca-9094-4fd8-8069-b91895d20613-istiod-ca-cert\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.949996 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.949952 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-envoy\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.949996 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.949985 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-credential-socket\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950280 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950006 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-istio-token\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950280 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-data\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950280 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cgbr\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-kube-api-access-9cgbr\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950280 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950108 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-socket\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950280 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950133 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83da9ca-9094-4fd8-8069-b91895d20613-istio-podinfo\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950280 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950163 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-certs\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950582 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950473 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-credential-socket\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950582 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950533 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-socket\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950582 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950563 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-certs\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950744 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950712 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-data\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.950789 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.950746 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83da9ca-9094-4fd8-8069-b91895d20613-istiod-ca-cert\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.952227 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.952206 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-envoy\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.952531 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.952512 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83da9ca-9094-4fd8-8069-b91895d20613-istio-podinfo\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.970688 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.970665 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-istio-token\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:03.976884 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:03.976864 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cgbr\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-kube-api-access-9cgbr\") pod \"e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:04.275075 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:04.275042 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:04.421528 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:04.421498 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq"] Jul 15 21:24:04.423010 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:24:04.422989 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83da9ca_9094_4fd8_8069_b91895d20613.slice/crio-f301c74a42cd13876552144071df45ff8cfe67811fb8c85e16675bac796ea17b WatchSource:0}: Error finding container f301c74a42cd13876552144071df45ff8cfe67811fb8c85e16675bac796ea17b: Status 404 returned error can't find the container with id f301c74a42cd13876552144071df45ff8cfe67811fb8c85e16675bac796ea17b Jul 15 21:24:04.425097 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:04.425060 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:24:04.425191 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:04.425142 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:24:04.425242 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:04.425197 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:24:04.858045 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:04.857997 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" event={"ID":"b83da9ca-9094-4fd8-8069-b91895d20613","Type":"ContainerStarted","Data":"9cde0fde4ffc1bf4d494a2a2d1c1a7348b8415773a338a0a37284c6ea7465be9"} Jul 15 21:24:04.858541 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:04.858076 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" event={"ID":"b83da9ca-9094-4fd8-8069-b91895d20613","Type":"ContainerStarted","Data":"f301c74a42cd13876552144071df45ff8cfe67811fb8c85e16675bac796ea17b"} Jul 15 21:24:04.893382 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:04.893332 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" podStartSLOduration=1.893314546 podStartE2EDuration="1.893314546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:24:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:24:04.891132558 +0000 UTC m=+1497.981365731" watchObservedRunningTime="2026-07-15 21:24:04.893314546 +0000 UTC m=+1497.983547795" Jul 15 21:24:05.275247 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:05.275207 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:05.280335 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:05.280309 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:05.863924 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:05.863895 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:05.865033 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:05.865008 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:07.490903 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:07.490806 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:24:07.501609 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:07.493088 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:24:07.501609 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:07.494829 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:24:07.501609 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:07.496835 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:24:16.930464 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:16.930428 2573 generic.go:356] "Generic (PLEG): container finished" podID="b83da9ca-9094-4fd8-8069-b91895d20613" containerID="9cde0fde4ffc1bf4d494a2a2d1c1a7348b8415773a338a0a37284c6ea7465be9" exitCode=0 Jul 15 21:24:16.930888 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:16.930498 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" event={"ID":"b83da9ca-9094-4fd8-8069-b91895d20613","Type":"ContainerDied","Data":"9cde0fde4ffc1bf4d494a2a2d1c1a7348b8415773a338a0a37284c6ea7465be9"} Jul 15 21:24:16.930946 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:16.930907 2573 scope.go:117] "RemoveContainer" containerID="9cde0fde4ffc1bf4d494a2a2d1c1a7348b8415773a338a0a37284c6ea7465be9" Jul 15 21:24:16.932003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:16.931969 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:24:16.932078 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:16.932062 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:24:16.932152 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:16.932109 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:24:17.943306 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:17.943253 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" event={"ID":"b83da9ca-9094-4fd8-8069-b91895d20613","Type":"ContainerStarted","Data":"de1384d869098ec40a65107236f7272668d5f54528a4bae651c7e6b3072b420d"} Jul 15 21:24:18.275557 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:18.275521 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:18.276595 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:18.276570 2573 patch_prober.go:28] interesting pod/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 21:24:18.276723 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:18.276631 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 21:24:19.276141 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:19.276104 2573 patch_prober.go:28] interesting pod/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 21:24:19.276650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:19.276194 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 21:24:20.275524 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:20.275485 2573 patch_prober.go:28] interesting pod/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 21:24:20.275695 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:20.275546 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 21:24:20.864715 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:20.864674 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:21.276140 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:21.276105 2573 patch_prober.go:28] interesting pod/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 21:24:21.276371 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:21.276156 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 21:24:22.280296 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:22.280265 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:24:22.281258 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:24:22.281237 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:25:25.179708 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:25.179672 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq"] Jul 15 21:25:25.180154 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:25.179909 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" containerID="cri-o://de1384d869098ec40a65107236f7272668d5f54528a4bae651c7e6b3072b420d" gracePeriod=30 Jul 15 21:25:30.360027 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.359991 2573 generic.go:356] "Generic (PLEG): container finished" podID="b83da9ca-9094-4fd8-8069-b91895d20613" containerID="de1384d869098ec40a65107236f7272668d5f54528a4bae651c7e6b3072b420d" exitCode=0 Jul 15 21:25:30.360383 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.360086 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" event={"ID":"b83da9ca-9094-4fd8-8069-b91895d20613","Type":"ContainerDied","Data":"de1384d869098ec40a65107236f7272668d5f54528a4bae651c7e6b3072b420d"} Jul 15 21:25:30.360383 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.360186 2573 scope.go:117] "RemoveContainer" containerID="9cde0fde4ffc1bf4d494a2a2d1c1a7348b8415773a338a0a37284c6ea7465be9" Jul 15 21:25:30.423501 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.423476 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:25:30.497557 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.497527 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-socket\") pod \"b83da9ca-9094-4fd8-8069-b91895d20613\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " Jul 15 21:25:30.497719 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.497643 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83da9ca-9094-4fd8-8069-b91895d20613-istio-podinfo\") pod \"b83da9ca-9094-4fd8-8069-b91895d20613\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " Jul 15 21:25:30.497719 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.497674 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cgbr\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-kube-api-access-9cgbr\") pod \"b83da9ca-9094-4fd8-8069-b91895d20613\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " Jul 15 21:25:30.497719 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.497707 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-envoy\") pod \"b83da9ca-9094-4fd8-8069-b91895d20613\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " Jul 15 21:25:30.497912 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.497838 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b83da9ca-9094-4fd8-8069-b91895d20613" (UID: "b83da9ca-9094-4fd8-8069-b91895d20613"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:25:30.497912 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.497897 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-credential-socket\") pod \"b83da9ca-9094-4fd8-8069-b91895d20613\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " Jul 15 21:25:30.498019 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.497961 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-certs\") pod \"b83da9ca-9094-4fd8-8069-b91895d20613\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " Jul 15 21:25:30.498019 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.497999 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-istio-token\") pod \"b83da9ca-9094-4fd8-8069-b91895d20613\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " Jul 15 21:25:30.498164 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.498021 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-data\") pod \"b83da9ca-9094-4fd8-8069-b91895d20613\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " Jul 15 21:25:30.498164 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.498044 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83da9ca-9094-4fd8-8069-b91895d20613-istiod-ca-cert\") pod \"b83da9ca-9094-4fd8-8069-b91895d20613\" (UID: \"b83da9ca-9094-4fd8-8069-b91895d20613\") " Jul 15 21:25:30.498342 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.498290 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b83da9ca-9094-4fd8-8069-b91895d20613" (UID: "b83da9ca-9094-4fd8-8069-b91895d20613"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:25:30.498661 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.498637 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:25:30.498661 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.498662 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:25:30.498911 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.498713 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b83da9ca-9094-4fd8-8069-b91895d20613-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b83da9ca-9094-4fd8-8069-b91895d20613" (UID: "b83da9ca-9094-4fd8-8069-b91895d20613"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:25:30.499642 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.499593 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-data" (OuterVolumeSpecName: "istio-data") pod "b83da9ca-9094-4fd8-8069-b91895d20613" (UID: "b83da9ca-9094-4fd8-8069-b91895d20613"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:25:30.499750 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.499665 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b83da9ca-9094-4fd8-8069-b91895d20613" (UID: "b83da9ca-9094-4fd8-8069-b91895d20613"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:25:30.501066 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.501040 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b83da9ca-9094-4fd8-8069-b91895d20613-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b83da9ca-9094-4fd8-8069-b91895d20613" (UID: "b83da9ca-9094-4fd8-8069-b91895d20613"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:25:30.501200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.501163 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-kube-api-access-9cgbr" (OuterVolumeSpecName: "kube-api-access-9cgbr") pod "b83da9ca-9094-4fd8-8069-b91895d20613" (UID: "b83da9ca-9094-4fd8-8069-b91895d20613"). InnerVolumeSpecName "kube-api-access-9cgbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:25:30.501386 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.501370 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b83da9ca-9094-4fd8-8069-b91895d20613" (UID: "b83da9ca-9094-4fd8-8069-b91895d20613"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:25:30.501496 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.501477 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-istio-token" (OuterVolumeSpecName: "istio-token") pod "b83da9ca-9094-4fd8-8069-b91895d20613" (UID: "b83da9ca-9094-4fd8-8069-b91895d20613"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:25:30.600072 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.600035 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:25:30.600072 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.600067 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:25:30.600072 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.600076 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:25:30.600072 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.600083 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b83da9ca-9094-4fd8-8069-b91895d20613-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:25:30.600406 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.600091 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b83da9ca-9094-4fd8-8069-b91895d20613-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:25:30.600406 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.600099 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cgbr\" (UniqueName: \"kubernetes.io/projected/b83da9ca-9094-4fd8-8069-b91895d20613-kube-api-access-9cgbr\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:25:30.600406 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:30.600107 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b83da9ca-9094-4fd8-8069-b91895d20613-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:25:31.367123 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:31.367093 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" Jul 15 21:25:31.367573 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:31.367137 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq" event={"ID":"b83da9ca-9094-4fd8-8069-b91895d20613","Type":"ContainerDied","Data":"f301c74a42cd13876552144071df45ff8cfe67811fb8c85e16675bac796ea17b"} Jul 15 21:25:31.367573 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:31.367207 2573 scope.go:117] "RemoveContainer" containerID="de1384d869098ec40a65107236f7272668d5f54528a4bae651c7e6b3072b420d" Jul 15 21:25:31.395134 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:31.395109 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq"] Jul 15 21:25:31.408261 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:31.408237 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0b8f8143-openshift-default-6b844b579-tbrcq"] Jul 15 21:25:31.449562 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:31.449529 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" path="/var/lib/kubelet/pods/b83da9ca-9094-4fd8-8069-b91895d20613/volumes" Jul 15 21:25:32.885894 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:32.885864 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr"] Jul 15 21:25:32.886318 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:32.886308 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" Jul 15 21:25:32.886318 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:32.886320 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" Jul 15 21:25:32.886426 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:32.886349 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" Jul 15 21:25:32.886426 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:32.886358 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" Jul 15 21:25:32.886426 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:32.886419 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" Jul 15 21:25:32.890414 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:32.890196 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:32.893639 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:32.893620 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-52019953-gw-openshift-default-dockercfg-28hd7\"" Jul 15 21:25:32.908910 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:32.908886 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr"] Jul 15 21:25:33.022918 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.022883 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-data\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.023104 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.022933 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istiod-ca-cert\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.023104 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.023030 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-certs\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.023104 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.023102 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-token\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.023251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.023134 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-credential-socket\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.023251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.023161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc4rk\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-kube-api-access-fc4rk\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.023251 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.023239 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-socket\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.023373 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.023279 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-podinfo\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.023373 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.023300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-envoy\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.124641 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.124610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-credential-socket\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.124832 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.124656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc4rk\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-kube-api-access-fc4rk\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.124832 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.124698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-socket\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.124832 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.124735 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-podinfo\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.124832 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.124762 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-envoy\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.124832 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.124802 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-data\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.125062 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.124836 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istiod-ca-cert\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.125062 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.124883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-certs\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.125062 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.124965 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-token\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.125062 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.125021 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-credential-socket\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.125260 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.125215 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-socket\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.125362 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.125340 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-data\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.125521 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.125495 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-certs\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.125714 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.125694 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istiod-ca-cert\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.127009 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.126990 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-envoy\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.127259 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.127242 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-podinfo\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.133572 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.133551 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-token\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.133827 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.133807 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc4rk\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-kube-api-access-fc4rk\") pod \"e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.200491 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.200394 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:33.343117 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.343093 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr"] Jul 15 21:25:33.344244 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:25:33.344219 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e7dee1_97b3_4424_b5c4_5b917ba07a55.slice/crio-f3147b724af2cbd4eaec9072adf8c26a4421d95867688708c03a55bbdd3d3f14 WatchSource:0}: Error finding container f3147b724af2cbd4eaec9072adf8c26a4421d95867688708c03a55bbdd3d3f14: Status 404 returned error can't find the container with id f3147b724af2cbd4eaec9072adf8c26a4421d95867688708c03a55bbdd3d3f14 Jul 15 21:25:33.346858 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.346825 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:25:33.346948 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.346891 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:25:33.346948 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.346927 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:25:33.381796 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:33.381709 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" event={"ID":"35e7dee1-97b3-4424-b5c4-5b917ba07a55","Type":"ContainerStarted","Data":"f3147b724af2cbd4eaec9072adf8c26a4421d95867688708c03a55bbdd3d3f14"} Jul 15 21:25:34.389713 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:34.389674 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" event={"ID":"35e7dee1-97b3-4424-b5c4-5b917ba07a55","Type":"ContainerStarted","Data":"8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35"} Jul 15 21:25:34.416981 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:34.416934 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" podStartSLOduration=2.416917839 podStartE2EDuration="2.416917839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:25:34.414248798 +0000 UTC m=+1587.504481979" watchObservedRunningTime="2026-07-15 21:25:34.416917839 +0000 UTC m=+1587.507151018" Jul 15 21:25:35.200697 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:35.200660 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:35.205516 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:35.205492 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:35.396679 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:35.396648 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:35.397799 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:35.397781 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:45.458814 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:45.458780 2573 generic.go:356] "Generic (PLEG): container finished" podID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerID="8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35" exitCode=0 Jul 15 21:25:45.459242 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:45.458854 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" event={"ID":"35e7dee1-97b3-4424-b5c4-5b917ba07a55","Type":"ContainerDied","Data":"8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35"} Jul 15 21:25:45.459242 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:45.459239 2573 scope.go:117] "RemoveContainer" containerID="8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35" Jul 15 21:25:45.460145 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:45.460118 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:25:45.460224 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:45.460210 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:25:45.460267 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:45.460249 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:25:46.467667 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:46.467630 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" event={"ID":"35e7dee1-97b3-4424-b5c4-5b917ba07a55","Type":"ContainerStarted","Data":"3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0"} Jul 15 21:25:47.200896 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:47.200859 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:47.202087 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:47.202067 2573 patch_prober.go:28] interesting pod/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 21:25:47.202148 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:47.202113 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 21:25:48.201048 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:48.201013 2573 patch_prober.go:28] interesting pod/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 21:25:48.201504 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:48.201080 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 21:25:49.201254 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:49.201218 2573 patch_prober.go:28] interesting pod/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 21:25:49.201639 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:49.201302 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 21:25:50.200969 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:50.200935 2573 patch_prober.go:28] interesting pod/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 21:25:50.201149 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:50.200992 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 21:25:50.397052 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:50.397024 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:51.205033 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:51.205004 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:25:51.206133 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:25:51.206112 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:26:38.760755 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:38.760709 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr"] Jul 15 21:26:38.761481 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:38.760999 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" containerID="cri-o://3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0" gracePeriod=30 Jul 15 21:26:44.012457 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.012426 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:26:44.111438 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111409 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-certs\") pod \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " Jul 15 21:26:44.111601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111446 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-envoy\") pod \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " Jul 15 21:26:44.111601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111478 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-socket\") pod \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " Jul 15 21:26:44.111601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111503 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-token\") pod \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " Jul 15 21:26:44.111601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111522 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-credential-socket\") pod \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " Jul 15 21:26:44.111601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111548 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-data\") pod \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " Jul 15 21:26:44.111601 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111567 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fc4rk\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-kube-api-access-fc4rk\") pod \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " Jul 15 21:26:44.111905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111643 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istiod-ca-cert\") pod \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " Jul 15 21:26:44.111905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111683 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-podinfo\") pod \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\" (UID: \"35e7dee1-97b3-4424-b5c4-5b917ba07a55\") " Jul 15 21:26:44.111905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111727 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "35e7dee1-97b3-4424-b5c4-5b917ba07a55" (UID: "35e7dee1-97b3-4424-b5c4-5b917ba07a55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:26:44.111905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111819 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "35e7dee1-97b3-4424-b5c4-5b917ba07a55" (UID: "35e7dee1-97b3-4424-b5c4-5b917ba07a55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:26:44.112147 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.111980 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "35e7dee1-97b3-4424-b5c4-5b917ba07a55" (UID: "35e7dee1-97b3-4424-b5c4-5b917ba07a55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:26:44.112147 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.112065 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:26:44.112147 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.112084 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:26:44.112147 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.112100 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:26:44.112147 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.112081 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "35e7dee1-97b3-4424-b5c4-5b917ba07a55" (UID: "35e7dee1-97b3-4424-b5c4-5b917ba07a55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:26:44.112523 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.112495 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-data" (OuterVolumeSpecName: "istio-data") pod "35e7dee1-97b3-4424-b5c4-5b917ba07a55" (UID: "35e7dee1-97b3-4424-b5c4-5b917ba07a55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:26:44.113987 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.113960 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "35e7dee1-97b3-4424-b5c4-5b917ba07a55" (UID: "35e7dee1-97b3-4424-b5c4-5b917ba07a55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:26:44.114327 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.114310 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "35e7dee1-97b3-4424-b5c4-5b917ba07a55" (UID: "35e7dee1-97b3-4424-b5c4-5b917ba07a55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:26:44.114489 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.114471 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-token" (OuterVolumeSpecName: "istio-token") pod "35e7dee1-97b3-4424-b5c4-5b917ba07a55" (UID: "35e7dee1-97b3-4424-b5c4-5b917ba07a55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:26:44.114489 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.114472 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-kube-api-access-fc4rk" (OuterVolumeSpecName: "kube-api-access-fc4rk") pod "35e7dee1-97b3-4424-b5c4-5b917ba07a55" (UID: "35e7dee1-97b3-4424-b5c4-5b917ba07a55"). InnerVolumeSpecName "kube-api-access-fc4rk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:26:44.212607 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.212577 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:26:44.212607 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.212600 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:26:44.212607 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.212609 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:26:44.212827 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.212617 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fc4rk\" (UniqueName: \"kubernetes.io/projected/35e7dee1-97b3-4424-b5c4-5b917ba07a55-kube-api-access-fc4rk\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:26:44.212827 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.212627 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:26:44.212827 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.212635 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35e7dee1-97b3-4424-b5c4-5b917ba07a55-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:26:44.809406 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.809373 2573 generic.go:356] "Generic (PLEG): container finished" podID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerID="3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0" exitCode=0 Jul 15 21:26:44.809600 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.809422 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" Jul 15 21:26:44.809600 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.809455 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" event={"ID":"35e7dee1-97b3-4424-b5c4-5b917ba07a55","Type":"ContainerDied","Data":"3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0"} Jul 15 21:26:44.809600 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.809513 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr" event={"ID":"35e7dee1-97b3-4424-b5c4-5b917ba07a55","Type":"ContainerDied","Data":"f3147b724af2cbd4eaec9072adf8c26a4421d95867688708c03a55bbdd3d3f14"} Jul 15 21:26:44.809600 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.809541 2573 scope.go:117] "RemoveContainer" containerID="3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0" Jul 15 21:26:44.819615 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.819597 2573 scope.go:117] "RemoveContainer" containerID="8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35" Jul 15 21:26:44.827774 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.827759 2573 scope.go:117] "RemoveContainer" containerID="3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0" Jul 15 21:26:44.828010 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:26:44.827992 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0\": container with ID starting with 3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0 not found: ID does not exist" containerID="3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0" Jul 15 21:26:44.828075 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.828015 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0"} err="failed to get container status \"3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0\": rpc error: code = NotFound desc = could not find container \"3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0\": container with ID starting with 3bbdd45054b6d16f396e7ff3352956b3e49b753940f9c2605b0cf147da5447a0 not found: ID does not exist" Jul 15 21:26:44.828075 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.828034 2573 scope.go:117] "RemoveContainer" containerID="8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35" Jul 15 21:26:44.828299 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:26:44.828282 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35\": container with ID starting with 8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35 not found: ID does not exist" containerID="8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35" Jul 15 21:26:44.828358 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.828302 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35"} err="failed to get container status \"8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35\": rpc error: code = NotFound desc = could not find container \"8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35\": container with ID starting with 8974009a23fe6cee04d83d98e064595e8ddee7ef8a0a00b0e2f2b3d08e389d35 not found: ID does not exist" Jul 15 21:26:44.838309 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.838136 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr"] Jul 15 21:26:44.844072 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:44.844041 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-52019953-gw-openshift-default-f7855646c-jjmrr"] Jul 15 21:26:45.448694 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:45.448657 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" path="/var/lib/kubelet/pods/35e7dee1-97b3-4424-b5c4-5b917ba07a55/volumes" Jul 15 21:26:51.716300 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.716271 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56"] Jul 15 21:26:51.716746 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.716732 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" Jul 15 21:26:51.716790 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.716748 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" Jul 15 21:26:51.716790 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.716755 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" Jul 15 21:26:51.716790 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.716761 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" Jul 15 21:26:51.716890 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.716819 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" Jul 15 21:26:51.716890 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.716828 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e7dee1-97b3-4424-b5c4-5b917ba07a55" containerName="istio-proxy" Jul 15 21:26:51.716890 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.716838 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b83da9ca-9094-4fd8-8069-b91895d20613" containerName="istio-proxy" Jul 15 21:26:51.719785 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.719764 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.723113 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.723095 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f1c9c155-openshift-default-dockercfg-b4q48\"" Jul 15 21:26:51.734816 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.734791 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56"] Jul 15 21:26:51.786378 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.786342 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb56f31-b73f-4b40-8253-620ec190e876-istio-podinfo\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.786378 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.786382 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-socket\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.786580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.786415 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbbsz\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-kube-api-access-nbbsz\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.786580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.786446 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-data\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.786580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.786466 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-credential-socket\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.786580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.786502 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-envoy\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.786580 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.786564 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-certs\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.786735 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.786598 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-istio-token\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.786735 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.786627 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb56f31-b73f-4b40-8253-620ec190e876-istiod-ca-cert\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888102 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888070 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb56f31-b73f-4b40-8253-620ec190e876-istio-podinfo\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888102 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888109 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-socket\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888380 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888136 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbbsz\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-kube-api-access-nbbsz\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888380 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888194 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-data\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888380 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888228 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-credential-socket\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888380 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888251 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-envoy\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888380 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888308 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-certs\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888380 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888356 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-istio-token\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888380 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888380 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb56f31-b73f-4b40-8253-620ec190e876-istiod-ca-cert\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888740 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888653 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-socket\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888740 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888674 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-credential-socket\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888740 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888654 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-data\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.888900 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.888736 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-certs\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.889041 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.889023 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb56f31-b73f-4b40-8253-620ec190e876-istiod-ca-cert\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.890457 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.890431 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-envoy\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.890911 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.890887 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb56f31-b73f-4b40-8253-620ec190e876-istio-podinfo\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.896661 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.896636 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-istio-token\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:51.896984 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:51.896961 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbbsz\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-kube-api-access-nbbsz\") pod \"e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:52.031401 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:52.031359 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:52.176747 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:52.176722 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56"] Jul 15 21:26:52.179041 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:26:52.179010 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb56f31_b73f_4b40_8253_620ec190e876.slice/crio-5ad4e17102d08059851e92ae2121c6ffc80b8bad9dff82112e4fdc39ee805a3a WatchSource:0}: Error finding container 5ad4e17102d08059851e92ae2121c6ffc80b8bad9dff82112e4fdc39ee805a3a: Status 404 returned error can't find the container with id 5ad4e17102d08059851e92ae2121c6ffc80b8bad9dff82112e4fdc39ee805a3a Jul 15 21:26:52.181264 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:52.181233 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:26:52.181358 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:52.181297 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:26:52.181358 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:52.181325 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:26:52.857698 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:52.857661 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" event={"ID":"6fb56f31-b73f-4b40-8253-620ec190e876","Type":"ContainerStarted","Data":"1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7"} Jul 15 21:26:52.858074 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:52.857719 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" event={"ID":"6fb56f31-b73f-4b40-8253-620ec190e876","Type":"ContainerStarted","Data":"5ad4e17102d08059851e92ae2121c6ffc80b8bad9dff82112e4fdc39ee805a3a"} Jul 15 21:26:52.882280 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:52.882231 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" podStartSLOduration=1.882216068 podStartE2EDuration="1.882216068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:26:52.879061999 +0000 UTC m=+1665.969295200" watchObservedRunningTime="2026-07-15 21:26:52.882216068 +0000 UTC m=+1665.972449246" Jul 15 21:26:53.032156 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:53.032126 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:53.037030 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:53.037006 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:53.863853 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:53.863825 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:26:53.864814 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:26:53.864792 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:27:03.924570 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:03.924464 2573 generic.go:356] "Generic (PLEG): container finished" podID="6fb56f31-b73f-4b40-8253-620ec190e876" containerID="1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7" exitCode=0 Jul 15 21:27:03.924570 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:03.924535 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" event={"ID":"6fb56f31-b73f-4b40-8253-620ec190e876","Type":"ContainerDied","Data":"1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7"} Jul 15 21:27:03.925009 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:03.924905 2573 scope.go:117] "RemoveContainer" containerID="1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7" Jul 15 21:27:03.925848 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:03.925825 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:27:03.925911 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:03.925886 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:27:03.925949 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:03.925919 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:27:04.934709 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:04.934666 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" event={"ID":"6fb56f31-b73f-4b40-8253-620ec190e876","Type":"ContainerStarted","Data":"1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3"} Jul 15 21:27:05.032248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:05.032214 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:27:05.033271 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:05.033248 2573 patch_prober.go:28] interesting pod/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 15 21:27:05.033476 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:05.033453 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 15 21:27:06.031830 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:06.031787 2573 patch_prober.go:28] interesting pod/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 15 21:27:06.032331 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:06.031852 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 15 21:27:07.040657 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:07.040610 2573 patch_prober.go:28] interesting pod/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 15 21:27:07.041156 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:07.040680 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 15 21:27:08.032025 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:08.031990 2573 patch_prober.go:28] interesting pod/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 15 21:27:08.032301 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:08.032272 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 15 21:27:08.864622 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:08.864582 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:27:09.036270 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:09.036239 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:27:09.037425 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:27:09.037402 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:28:04.500430 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:04.500387 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56"] Jul 15 21:28:04.501002 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:04.500673 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" containerID="cri-o://1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3" gracePeriod=30 Jul 15 21:28:09.039180 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.039140 2573 patch_prober.go:28] interesting pod/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 21:28:09.039638 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.039255 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 21:28:09.763341 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.763280 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:28:09.867413 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867384 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-socket\") pod \"6fb56f31-b73f-4b40-8253-620ec190e876\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " Jul 15 21:28:09.867565 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867432 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb56f31-b73f-4b40-8253-620ec190e876-istiod-ca-cert\") pod \"6fb56f31-b73f-4b40-8253-620ec190e876\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " Jul 15 21:28:09.867565 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867491 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbbsz\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-kube-api-access-nbbsz\") pod \"6fb56f31-b73f-4b40-8253-620ec190e876\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " Jul 15 21:28:09.867565 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867560 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-envoy\") pod \"6fb56f31-b73f-4b40-8253-620ec190e876\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " Jul 15 21:28:09.867734 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867585 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-certs\") pod \"6fb56f31-b73f-4b40-8253-620ec190e876\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " Jul 15 21:28:09.867734 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867634 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-data\") pod \"6fb56f31-b73f-4b40-8253-620ec190e876\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " Jul 15 21:28:09.867734 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867663 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-istio-token\") pod \"6fb56f31-b73f-4b40-8253-620ec190e876\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " Jul 15 21:28:09.867734 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867662 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6fb56f31-b73f-4b40-8253-620ec190e876" (UID: "6fb56f31-b73f-4b40-8253-620ec190e876"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:28:09.867734 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867710 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-credential-socket\") pod \"6fb56f31-b73f-4b40-8253-620ec190e876\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " Jul 15 21:28:09.868026 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867768 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb56f31-b73f-4b40-8253-620ec190e876-istio-podinfo\") pod \"6fb56f31-b73f-4b40-8253-620ec190e876\" (UID: \"6fb56f31-b73f-4b40-8253-620ec190e876\") " Jul 15 21:28:09.868026 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.867939 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fb56f31-b73f-4b40-8253-620ec190e876-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6fb56f31-b73f-4b40-8253-620ec190e876" (UID: "6fb56f31-b73f-4b40-8253-620ec190e876"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:28:09.868138 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.868113 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:28:09.868138 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.868134 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb56f31-b73f-4b40-8253-620ec190e876-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:28:09.868349 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.868226 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6fb56f31-b73f-4b40-8253-620ec190e876" (UID: "6fb56f31-b73f-4b40-8253-620ec190e876"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:28:09.868349 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.868304 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6fb56f31-b73f-4b40-8253-620ec190e876" (UID: "6fb56f31-b73f-4b40-8253-620ec190e876"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:28:09.868694 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.868672 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-data" (OuterVolumeSpecName: "istio-data") pod "6fb56f31-b73f-4b40-8253-620ec190e876" (UID: "6fb56f31-b73f-4b40-8253-620ec190e876"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:28:09.869984 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.869956 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6fb56f31-b73f-4b40-8253-620ec190e876" (UID: "6fb56f31-b73f-4b40-8253-620ec190e876"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:28:09.870097 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.870038 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-kube-api-access-nbbsz" (OuterVolumeSpecName: "kube-api-access-nbbsz") pod "6fb56f31-b73f-4b40-8253-620ec190e876" (UID: "6fb56f31-b73f-4b40-8253-620ec190e876"). InnerVolumeSpecName "kube-api-access-nbbsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:28:09.870352 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.870326 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-istio-token" (OuterVolumeSpecName: "istio-token") pod "6fb56f31-b73f-4b40-8253-620ec190e876" (UID: "6fb56f31-b73f-4b40-8253-620ec190e876"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:28:09.870450 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.870349 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6fb56f31-b73f-4b40-8253-620ec190e876-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6fb56f31-b73f-4b40-8253-620ec190e876" (UID: "6fb56f31-b73f-4b40-8253-620ec190e876"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:28:09.969397 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.969364 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:28:09.969397 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.969395 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb56f31-b73f-4b40-8253-620ec190e876-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:28:09.969397 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.969405 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbbsz\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-kube-api-access-nbbsz\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:28:09.969647 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.969416 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:28:09.969647 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.969425 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:28:09.969647 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.969432 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb56f31-b73f-4b40-8253-620ec190e876-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:28:09.969647 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:09.969440 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb56f31-b73f-4b40-8253-620ec190e876-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:28:10.330132 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.330093 2573 generic.go:356] "Generic (PLEG): container finished" podID="6fb56f31-b73f-4b40-8253-620ec190e876" containerID="1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3" exitCode=0 Jul 15 21:28:10.330616 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.330141 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" Jul 15 21:28:10.330616 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.330202 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" event={"ID":"6fb56f31-b73f-4b40-8253-620ec190e876","Type":"ContainerDied","Data":"1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3"} Jul 15 21:28:10.330616 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.330254 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56" event={"ID":"6fb56f31-b73f-4b40-8253-620ec190e876","Type":"ContainerDied","Data":"5ad4e17102d08059851e92ae2121c6ffc80b8bad9dff82112e4fdc39ee805a3a"} Jul 15 21:28:10.330616 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.330277 2573 scope.go:117] "RemoveContainer" containerID="1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3" Jul 15 21:28:10.342330 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.342295 2573 scope.go:117] "RemoveContainer" containerID="1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7" Jul 15 21:28:10.351216 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.351193 2573 scope.go:117] "RemoveContainer" containerID="1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3" Jul 15 21:28:10.351517 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:28:10.351492 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3\": container with ID starting with 1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3 not found: ID does not exist" containerID="1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3" Jul 15 21:28:10.351583 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.351529 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3"} err="failed to get container status \"1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3\": rpc error: code = NotFound desc = could not find container \"1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3\": container with ID starting with 1e7c285246434130cbbb4e042da7842c575009391fb4441a251d8bb327b460f3 not found: ID does not exist" Jul 15 21:28:10.351583 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.351552 2573 scope.go:117] "RemoveContainer" containerID="1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7" Jul 15 21:28:10.351971 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:28:10.351948 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7\": container with ID starting with 1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7 not found: ID does not exist" containerID="1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7" Jul 15 21:28:10.352052 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.351980 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7"} err="failed to get container status \"1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7\": rpc error: code = NotFound desc = could not find container \"1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7\": container with ID starting with 1a78deba5fd3f1064fcc0ca5b0103eb629b1a0f3273746ccf6d194be5ba618d7 not found: ID does not exist" Jul 15 21:28:10.355626 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.355599 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56"] Jul 15 21:28:10.358197 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:10.358161 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f1c9c155-openshift-default-5447b77d85-44n56"] Jul 15 21:28:11.447143 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.447110 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" path="/var/lib/kubelet/pods/6fb56f31-b73f-4b40-8253-620ec190e876/volumes" Jul 15 21:28:11.637942 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.637912 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm"] Jul 15 21:28:11.638556 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.638530 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" Jul 15 21:28:11.638556 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.638554 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" Jul 15 21:28:11.638758 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.638681 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" Jul 15 21:28:11.638758 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.638698 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" Jul 15 21:28:11.643813 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.643787 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.650383 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.650363 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ceeadc32-openshift-default-dockercfg-fdccg\"" Jul 15 21:28:11.661098 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.661071 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm"] Jul 15 21:28:11.786209 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.786153 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-istio-token\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.786209 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.786210 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-envoy\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.786439 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.786260 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-data\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.786439 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.786297 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb021052-a361-430b-bfe6-386daeeea727-istio-podinfo\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.786439 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.786363 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-credential-socket\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.786439 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.786412 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-socket\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.786592 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.786449 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb021052-a361-430b-bfe6-386daeeea727-istiod-ca-cert\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.786592 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.786484 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx8xf\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-kube-api-access-lx8xf\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.786592 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.786524 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-certs\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.887450 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887413 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb021052-a361-430b-bfe6-386daeeea727-istio-podinfo\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.887627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887467 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-credential-socket\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.887627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887514 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-socket\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.887627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887534 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb021052-a361-430b-bfe6-386daeeea727-istiod-ca-cert\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.887627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887555 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx8xf\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-kube-api-access-lx8xf\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.887627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887580 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-certs\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.887889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887724 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-istio-token\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.887889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887783 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-envoy\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.887889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-data\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.888060 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887963 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-credential-socket\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.888060 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.887984 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-socket\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.888164 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.888066 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-certs\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.888273 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.888237 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-data\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.888420 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.888400 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb021052-a361-430b-bfe6-386daeeea727-istiod-ca-cert\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.889984 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.889956 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-envoy\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.890094 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.890078 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb021052-a361-430b-bfe6-386daeeea727-istio-podinfo\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.898294 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.898269 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-istio-token\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.900748 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.900725 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx8xf\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-kube-api-access-lx8xf\") pod \"e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:11.955247 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:11.955217 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:12.104786 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:12.104759 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm"] Jul 15 21:28:12.105670 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:28:12.105643 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb021052_a361_430b_bfe6_386daeeea727.slice/crio-fca3968f99e26a013e85938f5830e4896b9333700dd8a1d64a5e33bdbc0d55d0 WatchSource:0}: Error finding container fca3968f99e26a013e85938f5830e4896b9333700dd8a1d64a5e33bdbc0d55d0: Status 404 returned error can't find the container with id fca3968f99e26a013e85938f5830e4896b9333700dd8a1d64a5e33bdbc0d55d0 Jul 15 21:28:12.107862 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:12.107834 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:28:12.107935 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:12.107892 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:28:12.107935 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:12.107920 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:28:12.344028 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:12.343949 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" event={"ID":"bb021052-a361-430b-bfe6-386daeeea727","Type":"ContainerStarted","Data":"63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c"} Jul 15 21:28:12.344028 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:12.344007 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" event={"ID":"bb021052-a361-430b-bfe6-386daeeea727","Type":"ContainerStarted","Data":"fca3968f99e26a013e85938f5830e4896b9333700dd8a1d64a5e33bdbc0d55d0"} Jul 15 21:28:12.370142 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:12.370086 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" podStartSLOduration=1.370069975 podStartE2EDuration="1.370069975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:28:12.368023624 +0000 UTC m=+1745.458256842" watchObservedRunningTime="2026-07-15 21:28:12.370069975 +0000 UTC m=+1745.460303152" Jul 15 21:28:12.955371 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:12.955335 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:13.961399 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:13.961370 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:14.362232 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:14.362201 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:14.363290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:14.363265 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:29.363661 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:29.363562 2573 patch_prober.go:28] interesting pod/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 15 21:28:29.363661 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:29.363622 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 15 21:28:29.458350 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:29.458316 2573 generic.go:356] "Generic (PLEG): container finished" podID="bb021052-a361-430b-bfe6-386daeeea727" containerID="63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c" exitCode=0 Jul 15 21:28:29.458489 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:29.458362 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" event={"ID":"bb021052-a361-430b-bfe6-386daeeea727","Type":"ContainerDied","Data":"63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c"} Jul 15 21:28:29.458743 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:29.458730 2573 scope.go:117] "RemoveContainer" containerID="63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c" Jul 15 21:28:29.459725 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:29.459696 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:28:29.459792 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:29.459765 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:28:29.459827 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:29.459800 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:28:30.466224 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:30.466184 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" event={"ID":"bb021052-a361-430b-bfe6-386daeeea727","Type":"ContainerStarted","Data":"4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9"} Jul 15 21:28:30.956145 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:30.956104 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:30.956394 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:30.956366 2573 patch_prober.go:28] interesting pod/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 15 21:28:30.956479 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:30.956426 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 15 21:28:31.956083 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:31.956049 2573 patch_prober.go:28] interesting pod/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 15 21:28:31.956522 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:31.956130 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 15 21:28:32.956010 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:32.955971 2573 patch_prober.go:28] interesting pod/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 15 21:28:32.956211 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:32.956025 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 15 21:28:33.956366 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:33.956330 2573 patch_prober.go:28] interesting pod/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 15 21:28:33.956796 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:33.956387 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 15 21:28:34.960324 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:34.960292 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:34.960838 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:34.960690 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:28:34.961465 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:28:34.961442 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:29:07.532912 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:07.532881 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:29:07.536161 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:07.536137 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:29:07.536877 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:07.536859 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:29:07.539969 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:07.539944 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:29:22.488057 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:22.488021 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm"] Jul 15 21:29:22.488466 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:22.488247 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" containerID="cri-o://4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9" gracePeriod=30 Jul 15 21:29:27.750523 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.750450 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:29:27.799615 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.799583 2573 generic.go:356] "Generic (PLEG): container finished" podID="bb021052-a361-430b-bfe6-386daeeea727" containerID="4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9" exitCode=0 Jul 15 21:29:27.799795 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.799641 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" Jul 15 21:29:27.799795 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.799667 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" event={"ID":"bb021052-a361-430b-bfe6-386daeeea727","Type":"ContainerDied","Data":"4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9"} Jul 15 21:29:27.799795 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.799736 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm" event={"ID":"bb021052-a361-430b-bfe6-386daeeea727","Type":"ContainerDied","Data":"fca3968f99e26a013e85938f5830e4896b9333700dd8a1d64a5e33bdbc0d55d0"} Jul 15 21:29:27.799795 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.799762 2573 scope.go:117] "RemoveContainer" containerID="4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9" Jul 15 21:29:27.811807 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.811783 2573 scope.go:117] "RemoveContainer" containerID="63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c" Jul 15 21:29:27.819340 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.819318 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-data\") pod \"bb021052-a361-430b-bfe6-386daeeea727\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " Jul 15 21:29:27.819462 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.819363 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb021052-a361-430b-bfe6-386daeeea727-istio-podinfo\") pod \"bb021052-a361-430b-bfe6-386daeeea727\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " Jul 15 21:29:27.819462 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.819412 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-credential-socket\") pod \"bb021052-a361-430b-bfe6-386daeeea727\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " Jul 15 21:29:27.819576 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.819477 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-socket\") pod \"bb021052-a361-430b-bfe6-386daeeea727\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " Jul 15 21:29:27.819576 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.819508 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-envoy\") pod \"bb021052-a361-430b-bfe6-386daeeea727\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " Jul 15 21:29:27.819576 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.819535 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-istio-token\") pod \"bb021052-a361-430b-bfe6-386daeeea727\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " Jul 15 21:29:27.819576 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.819558 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx8xf\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-kube-api-access-lx8xf\") pod \"bb021052-a361-430b-bfe6-386daeeea727\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " Jul 15 21:29:27.819771 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.819622 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb021052-a361-430b-bfe6-386daeeea727-istiod-ca-cert\") pod \"bb021052-a361-430b-bfe6-386daeeea727\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " Jul 15 21:29:27.819771 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.819657 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-certs\") pod \"bb021052-a361-430b-bfe6-386daeeea727\" (UID: \"bb021052-a361-430b-bfe6-386daeeea727\") " Jul 15 21:29:27.820346 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.820064 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb021052-a361-430b-bfe6-386daeeea727" (UID: "bb021052-a361-430b-bfe6-386daeeea727"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:29:27.820346 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.820189 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb021052-a361-430b-bfe6-386daeeea727" (UID: "bb021052-a361-430b-bfe6-386daeeea727"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:29:27.820501 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.820368 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb021052-a361-430b-bfe6-386daeeea727" (UID: "bb021052-a361-430b-bfe6-386daeeea727"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:29:27.820654 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.820631 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb021052-a361-430b-bfe6-386daeeea727" (UID: "bb021052-a361-430b-bfe6-386daeeea727"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:29:27.820922 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.820898 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb021052-a361-430b-bfe6-386daeeea727-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb021052-a361-430b-bfe6-386daeeea727" (UID: "bb021052-a361-430b-bfe6-386daeeea727"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:29:27.824920 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.824899 2573 scope.go:117] "RemoveContainer" containerID="4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9" Jul 15 21:29:27.826916 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:29:27.826768 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9\": container with ID starting with 4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9 not found: ID does not exist" containerID="4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9" Jul 15 21:29:27.826916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.826807 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9"} err="failed to get container status \"4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9\": rpc error: code = NotFound desc = could not find container \"4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9\": container with ID starting with 4291350ebd67487ddc39674c7aca0e9c3db62f55ebc63054caeb5872aa12c8c9 not found: ID does not exist" Jul 15 21:29:27.826916 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.826831 2573 scope.go:117] "RemoveContainer" containerID="63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c" Jul 15 21:29:27.827343 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.827280 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb021052-a361-430b-bfe6-386daeeea727" (UID: "bb021052-a361-430b-bfe6-386daeeea727"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:29:27.827343 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.827317 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-kube-api-access-lx8xf" (OuterVolumeSpecName: "kube-api-access-lx8xf") pod "bb021052-a361-430b-bfe6-386daeeea727" (UID: "bb021052-a361-430b-bfe6-386daeeea727"). InnerVolumeSpecName "kube-api-access-lx8xf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:29:27.827343 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.827325 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb021052-a361-430b-bfe6-386daeeea727-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb021052-a361-430b-bfe6-386daeeea727" (UID: "bb021052-a361-430b-bfe6-386daeeea727"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:29:27.827613 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.827564 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb021052-a361-430b-bfe6-386daeeea727" (UID: "bb021052-a361-430b-bfe6-386daeeea727"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:29:27.827694 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:29:27.827607 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c\": container with ID starting with 63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c not found: ID does not exist" containerID="63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c" Jul 15 21:29:27.827694 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.827639 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c"} err="failed to get container status \"63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c\": rpc error: code = NotFound desc = could not find container \"63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c\": container with ID starting with 63eaa897c846ff293ea398ade710a75c2bad3d5c70b5ef86ec4a04cb3d5f809c not found: ID does not exist" Jul 15 21:29:27.921048 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.921011 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:29:27.921048 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.921044 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:29:27.921282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.921059 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:29:27.921282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.921072 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx8xf\" (UniqueName: \"kubernetes.io/projected/bb021052-a361-430b-bfe6-386daeeea727-kube-api-access-lx8xf\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:29:27.921282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.921084 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb021052-a361-430b-bfe6-386daeeea727-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:29:27.921282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.921096 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:29:27.921282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.921107 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:29:27.921282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.921121 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb021052-a361-430b-bfe6-386daeeea727-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:29:27.921282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:27.921133 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb021052-a361-430b-bfe6-386daeeea727-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:29:28.124966 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:28.124929 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm"] Jul 15 21:29:28.130094 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:28.130068 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ceeadc32-openshift-default-646ff948f9-wzkcm"] Jul 15 21:29:29.448740 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:29:29.448709 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb021052-a361-430b-bfe6-386daeeea727" path="/var/lib/kubelet/pods/bb021052-a361-430b-bfe6-386daeeea727/volumes" Jul 15 21:31:10.653846 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.653809 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp"] Jul 15 21:31:10.654376 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.654334 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" Jul 15 21:31:10.654439 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.654378 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fb56f31-b73f-4b40-8253-620ec190e876" containerName="istio-proxy" Jul 15 21:31:10.654439 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.654407 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" Jul 15 21:31:10.654439 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.654415 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" Jul 15 21:31:10.654595 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.654516 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" Jul 15 21:31:10.657526 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.657506 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.660610 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.660592 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-77f212fb-openshift-default-dockercfg-tb6hr\"" Jul 15 21:31:10.671750 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.671728 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp"] Jul 15 21:31:10.792049 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.792013 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-envoy\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.792248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.792054 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-socket\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.792248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.792081 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f4652-d72f-4ecb-a406-748b1798d706-istio-podinfo\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.792248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.792109 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5vbb\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-kube-api-access-s5vbb\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.792248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.792140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-certs\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.792429 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.792239 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-data\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.792429 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.792289 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-credential-socket\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.792429 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.792319 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f4652-d72f-4ecb-a406-748b1798d706-istiod-ca-cert\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.792429 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.792381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-istio-token\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.893520 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.893489 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-envoy\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.893707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.893529 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-socket\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.893707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.893546 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f4652-d72f-4ecb-a406-748b1798d706-istio-podinfo\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.893707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.893566 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5vbb\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-kube-api-access-s5vbb\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.893707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.893595 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-certs\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.893707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.893637 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-data\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.893707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.893659 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-credential-socket\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.893707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.893685 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f4652-d72f-4ecb-a406-748b1798d706-istiod-ca-cert\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.894071 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.893949 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-istio-token\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.894129 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.894091 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-certs\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.894203 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.894122 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-data\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.894311 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.894289 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-socket\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.894371 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.894331 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-credential-socket\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.894632 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.894608 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f4652-d72f-4ecb-a406-748b1798d706-istiod-ca-cert\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.896338 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.896307 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-envoy\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.896490 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.896475 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f4652-d72f-4ecb-a406-748b1798d706-istio-podinfo\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.903808 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.903787 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-istio-token\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.904100 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.904050 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5vbb\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-kube-api-access-s5vbb\") pod \"e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:10.967701 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:10.967676 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:11.129415 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:11.129384 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp"] Jul 15 21:31:11.130129 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:31:11.130087 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45f4652_d72f_4ecb_a406_748b1798d706.slice/crio-cb6afed23b8a003930cfa884acb82af1fd591567378b550cd070390ca148476e WatchSource:0}: Error finding container cb6afed23b8a003930cfa884acb82af1fd591567378b550cd070390ca148476e: Status 404 returned error can't find the container with id cb6afed23b8a003930cfa884acb82af1fd591567378b550cd070390ca148476e Jul 15 21:31:11.132420 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:11.132380 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:31:11.132522 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:11.132460 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:31:11.132522 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:11.132499 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:31:11.403241 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:11.403204 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" event={"ID":"f45f4652-d72f-4ecb-a406-748b1798d706","Type":"ContainerStarted","Data":"9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8"} Jul 15 21:31:11.403415 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:11.403265 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" event={"ID":"f45f4652-d72f-4ecb-a406-748b1798d706","Type":"ContainerStarted","Data":"cb6afed23b8a003930cfa884acb82af1fd591567378b550cd070390ca148476e"} Jul 15 21:31:11.429610 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:11.429517 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" podStartSLOduration=1.4295024459999999 podStartE2EDuration="1.429502446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:31:11.426927853 +0000 UTC m=+1924.517161031" watchObservedRunningTime="2026-07-15 21:31:11.429502446 +0000 UTC m=+1924.519735626" Jul 15 21:31:11.968567 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:11.968529 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:12.972889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:12.972858 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:13.414442 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:13.414410 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:13.415634 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:13.415613 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:28.520926 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:28.520607 2573 patch_prober.go:28] interesting pod/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:50662->10.132.0.60:15021: read: connection reset by peer" start-of-body= Jul 15 21:31:28.520926 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:28.520665 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:50662->10.132.0.60:15021: read: connection reset by peer" Jul 15 21:31:29.516634 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:29.516600 2573 generic.go:356] "Generic (PLEG): container finished" podID="f45f4652-d72f-4ecb-a406-748b1798d706" containerID="9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8" exitCode=0 Jul 15 21:31:29.516814 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:29.516680 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" event={"ID":"f45f4652-d72f-4ecb-a406-748b1798d706","Type":"ContainerDied","Data":"9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8"} Jul 15 21:31:29.517111 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:29.517096 2573 scope.go:117] "RemoveContainer" containerID="9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8" Jul 15 21:31:29.518148 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:29.518115 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:31:29.518291 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:29.518215 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:31:29.518291 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:29.518257 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:31:30.525300 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:30.525266 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" event={"ID":"f45f4652-d72f-4ecb-a406-748b1798d706","Type":"ContainerStarted","Data":"425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8"} Jul 15 21:31:30.967926 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:30.967844 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:30.968077 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:30.967996 2573 patch_prober.go:28] interesting pod/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 15 21:31:30.968077 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:30.968035 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 15 21:31:31.968821 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:31.968787 2573 patch_prober.go:28] interesting pod/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 15 21:31:31.969237 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:31.968861 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 15 21:31:32.968261 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:32.968224 2573 patch_prober.go:28] interesting pod/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 15 21:31:32.968446 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:32.968278 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 15 21:31:33.968042 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:33.968005 2573 patch_prober.go:28] interesting pod/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 15 21:31:33.968465 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:33.968063 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 15 21:31:34.971738 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:34.971710 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:34.972217 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:34.972100 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:34.972706 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:34.972687 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:47.821526 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:47.821492 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp"] Jul 15 21:31:47.822079 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:47.821748 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" containerID="cri-o://425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8" gracePeriod=30 Jul 15 21:31:49.974802 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:49.974762 2573 patch_prober.go:28] interesting pod/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 15 21:31:49.975217 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:49.974830 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 21:31:53.069735 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.069707 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:53.113990 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.113908 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f4652-d72f-4ecb-a406-748b1798d706-istiod-ca-cert\") pod \"f45f4652-d72f-4ecb-a406-748b1798d706\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " Jul 15 21:31:53.114120 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.113999 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-data\") pod \"f45f4652-d72f-4ecb-a406-748b1798d706\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " Jul 15 21:31:53.114120 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114077 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-certs\") pod \"f45f4652-d72f-4ecb-a406-748b1798d706\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " Jul 15 21:31:53.114120 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114111 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-istio-token\") pod \"f45f4652-d72f-4ecb-a406-748b1798d706\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " Jul 15 21:31:53.114321 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114145 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-credential-socket\") pod \"f45f4652-d72f-4ecb-a406-748b1798d706\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " Jul 15 21:31:53.114381 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114299 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5vbb\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-kube-api-access-s5vbb\") pod \"f45f4652-d72f-4ecb-a406-748b1798d706\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " Jul 15 21:31:53.114381 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114340 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f45f4652-d72f-4ecb-a406-748b1798d706-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f45f4652-d72f-4ecb-a406-748b1798d706" (UID: "f45f4652-d72f-4ecb-a406-748b1798d706"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:31:53.114489 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114390 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f4652-d72f-4ecb-a406-748b1798d706-istio-podinfo\") pod \"f45f4652-d72f-4ecb-a406-748b1798d706\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " Jul 15 21:31:53.114547 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114504 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-envoy\") pod \"f45f4652-d72f-4ecb-a406-748b1798d706\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " Jul 15 21:31:53.114547 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114443 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f45f4652-d72f-4ecb-a406-748b1798d706" (UID: "f45f4652-d72f-4ecb-a406-748b1798d706"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:53.114547 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114536 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-socket\") pod \"f45f4652-d72f-4ecb-a406-748b1798d706\" (UID: \"f45f4652-d72f-4ecb-a406-748b1798d706\") " Jul 15 21:31:53.114805 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114783 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f45f4652-d72f-4ecb-a406-748b1798d706" (UID: "f45f4652-d72f-4ecb-a406-748b1798d706"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:53.114889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114855 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f45f4652-d72f-4ecb-a406-748b1798d706" (UID: "f45f4652-d72f-4ecb-a406-748b1798d706"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:53.114947 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114913 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:31:53.114947 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114933 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:31:53.115053 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.114948 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f45f4652-d72f-4ecb-a406-748b1798d706-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:31:53.115492 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.115467 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-data" (OuterVolumeSpecName: "istio-data") pod "f45f4652-d72f-4ecb-a406-748b1798d706" (UID: "f45f4652-d72f-4ecb-a406-748b1798d706"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:53.116754 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.116730 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f45f4652-d72f-4ecb-a406-748b1798d706-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f45f4652-d72f-4ecb-a406-748b1798d706" (UID: "f45f4652-d72f-4ecb-a406-748b1798d706"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:31:53.116866 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.116840 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f45f4652-d72f-4ecb-a406-748b1798d706" (UID: "f45f4652-d72f-4ecb-a406-748b1798d706"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:31:53.116930 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.116856 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-kube-api-access-s5vbb" (OuterVolumeSpecName: "kube-api-access-s5vbb") pod "f45f4652-d72f-4ecb-a406-748b1798d706" (UID: "f45f4652-d72f-4ecb-a406-748b1798d706"). InnerVolumeSpecName "kube-api-access-s5vbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:31:53.117047 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.117033 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-istio-token" (OuterVolumeSpecName: "istio-token") pod "f45f4652-d72f-4ecb-a406-748b1798d706" (UID: "f45f4652-d72f-4ecb-a406-748b1798d706"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:31:53.216418 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.216368 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:31:53.216418 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.216411 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:31:53.216418 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.216422 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:31:53.216418 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.216429 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5vbb\" (UniqueName: \"kubernetes.io/projected/f45f4652-d72f-4ecb-a406-748b1798d706-kube-api-access-s5vbb\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:31:53.216418 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.216439 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f45f4652-d72f-4ecb-a406-748b1798d706-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:31:53.216735 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.216447 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f45f4652-d72f-4ecb-a406-748b1798d706-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:31:53.666942 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.666904 2573 generic.go:356] "Generic (PLEG): container finished" podID="f45f4652-d72f-4ecb-a406-748b1798d706" containerID="425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8" exitCode=0 Jul 15 21:31:53.667144 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.666983 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" Jul 15 21:31:53.667144 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.666983 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" event={"ID":"f45f4652-d72f-4ecb-a406-748b1798d706","Type":"ContainerDied","Data":"425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8"} Jul 15 21:31:53.667144 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.667096 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp" event={"ID":"f45f4652-d72f-4ecb-a406-748b1798d706","Type":"ContainerDied","Data":"cb6afed23b8a003930cfa884acb82af1fd591567378b550cd070390ca148476e"} Jul 15 21:31:53.667144 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.667118 2573 scope.go:117] "RemoveContainer" containerID="425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8" Jul 15 21:31:53.676217 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.676203 2573 scope.go:117] "RemoveContainer" containerID="9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8" Jul 15 21:31:53.684477 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.684456 2573 scope.go:117] "RemoveContainer" containerID="425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8" Jul 15 21:31:53.684747 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:31:53.684729 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8\": container with ID starting with 425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8 not found: ID does not exist" containerID="425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8" Jul 15 21:31:53.684799 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.684755 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8"} err="failed to get container status \"425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8\": rpc error: code = NotFound desc = could not find container \"425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8\": container with ID starting with 425cb0f89f16d54000a5c554b0586abee07d05b1d0e1bf6d4136e8ce14712ce8 not found: ID does not exist" Jul 15 21:31:53.684799 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.684771 2573 scope.go:117] "RemoveContainer" containerID="9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8" Jul 15 21:31:53.685000 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:31:53.684984 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8\": container with ID starting with 9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8 not found: ID does not exist" containerID="9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8" Jul 15 21:31:53.685054 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.685006 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8"} err="failed to get container status \"9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8\": rpc error: code = NotFound desc = could not find container \"9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8\": container with ID starting with 9c9bc2e18b788b76d3cd5c3c7f17befd81ad0a4332f3da6b2662be2070f317d8 not found: ID does not exist" Jul 15 21:31:53.689512 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.689494 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp"] Jul 15 21:31:53.693578 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:53.693559 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-77f212fb-openshift-default-86bcd9ff75-mstkp"] Jul 15 21:31:55.447951 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:31:55.447908 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" path="/var/lib/kubelet/pods/f45f4652-d72f-4ecb-a406-748b1798d706/volumes" Jul 15 21:32:08.118344 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118312 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9"] Jul 15 21:32:08.118732 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118720 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" Jul 15 21:32:08.118777 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118734 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" Jul 15 21:32:08.118777 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118745 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" Jul 15 21:32:08.118777 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118751 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" Jul 15 21:32:08.118777 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118771 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" Jul 15 21:32:08.118777 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118778 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" Jul 15 21:32:08.118929 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118838 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb021052-a361-430b-bfe6-386daeeea727" containerName="istio-proxy" Jul 15 21:32:08.118929 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118845 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" Jul 15 21:32:08.118929 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.118852 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f45f4652-d72f-4ecb-a406-748b1798d706" containerName="istio-proxy" Jul 15 21:32:08.122210 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.122157 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.125157 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.125136 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-372e1c00-openshift-default-dockercfg-49b5s\"" Jul 15 21:32:08.136631 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.136606 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9"] Jul 15 21:32:08.268416 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.268373 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-credential-socket\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.268596 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.268429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-socket\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.268596 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.268489 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-envoy\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.268596 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.268551 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istiod-ca-cert\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.268596 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.268589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-token\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.268736 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.268610 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-certs\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.268736 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.268710 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-podinfo\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.268736 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.268727 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccfq9\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-kube-api-access-ccfq9\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.268829 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.268772 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-data\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.369778 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.369688 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-podinfo\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.369778 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.369732 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccfq9\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-kube-api-access-ccfq9\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.369778 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.369775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-data\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370044 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.369804 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-credential-socket\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370044 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.369842 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-socket\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370044 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.369870 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-envoy\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370044 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.369966 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istiod-ca-cert\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370044 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.370027 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-token\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.370063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-certs\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.370273 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-credential-socket\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.370298 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-socket\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370540 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.370456 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-data\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370540 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.370524 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-certs\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.370766 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.370746 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istiod-ca-cert\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.372025 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.372003 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-envoy\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.372364 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.372342 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-podinfo\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.379310 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.379289 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-token\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.379509 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.379490 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccfq9\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-kube-api-access-ccfq9\") pod \"e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.433290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.433245 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:08.578009 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.577987 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9"] Jul 15 21:32:08.579526 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:32:08.579491 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4232ba_9cf9_46e9_9cf7_743515cb7155.slice/crio-7a1c1c9f3ad914ad067c9fe55e3b9a4f1b5a86b3866fd65566109a734d2c10ab WatchSource:0}: Error finding container 7a1c1c9f3ad914ad067c9fe55e3b9a4f1b5a86b3866fd65566109a734d2c10ab: Status 404 returned error can't find the container with id 7a1c1c9f3ad914ad067c9fe55e3b9a4f1b5a86b3866fd65566109a734d2c10ab Jul 15 21:32:08.581504 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.581472 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:32:08.581603 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.581548 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:32:08.581603 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.581596 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:32:08.761482 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.761443 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" event={"ID":"bf4232ba-9cf9-46e9-9cf7-743515cb7155","Type":"ContainerStarted","Data":"b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7"} Jul 15 21:32:08.761678 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.761507 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" event={"ID":"bf4232ba-9cf9-46e9-9cf7-743515cb7155","Type":"ContainerStarted","Data":"7a1c1c9f3ad914ad067c9fe55e3b9a4f1b5a86b3866fd65566109a734d2c10ab"} Jul 15 21:32:08.785839 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:08.785781 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" podStartSLOduration=0.78576486 podStartE2EDuration="785.76486ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:32:08.783985933 +0000 UTC m=+1981.874219113" watchObservedRunningTime="2026-07-15 21:32:08.78576486 +0000 UTC m=+1981.875998038" Jul 15 21:32:09.434471 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:09.434437 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:09.439918 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:09.439892 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:09.767138 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:09.767105 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:09.768158 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:09.768141 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:25.201831 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:25.201078 2573 patch_prober.go:28] interesting pod/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:55628->10.132.0.61:15021: read: connection reset by peer" start-of-body= Jul 15 21:32:25.201831 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:25.201160 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:55628->10.132.0.61:15021: read: connection reset by peer" Jul 15 21:32:25.870840 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:25.870808 2573 generic.go:356] "Generic (PLEG): container finished" podID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerID="b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7" exitCode=0 Jul 15 21:32:25.871013 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:25.870877 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" event={"ID":"bf4232ba-9cf9-46e9-9cf7-743515cb7155","Type":"ContainerDied","Data":"b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7"} Jul 15 21:32:25.871287 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:25.871272 2573 scope.go:117] "RemoveContainer" containerID="b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7" Jul 15 21:32:25.872282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:25.872257 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:32:25.872347 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:25.872323 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:32:25.872383 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:25.872363 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:32:26.879032 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:26.878998 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" event={"ID":"bf4232ba-9cf9-46e9-9cf7-743515cb7155","Type":"ContainerStarted","Data":"e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729"} Jul 15 21:32:27.433918 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:27.433871 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:27.434135 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:27.434040 2573 patch_prober.go:28] interesting pod/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 15 21:32:27.434135 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:27.434099 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 15 21:32:28.434448 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:28.434409 2573 patch_prober.go:28] interesting pod/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 15 21:32:28.434923 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:28.434481 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 15 21:32:29.441610 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:29.441563 2573 patch_prober.go:28] interesting pod/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 15 21:32:29.442067 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:29.441641 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 15 21:32:30.434703 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:30.434652 2573 patch_prober.go:28] interesting pod/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 15 21:32:30.434915 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:30.434727 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 15 21:32:31.437141 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:31.437103 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:31.437569 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:31.437521 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:32:31.438154 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:32:31.438137 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:34:07.576732 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:07.576706 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:34:07.580061 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:07.580034 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:34:07.580557 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:07.580531 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:34:07.583835 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:07.583818 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:34:15.248547 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:15.248507 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9"] Jul 15 21:34:15.249033 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:15.248730 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" containerID="cri-o://e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729" gracePeriod=30 Jul 15 21:34:16.440097 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:16.440058 2573 patch_prober.go:28] interesting pod/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 21:34:16.440490 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:16.440125 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 21:34:20.509093 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.509033 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:34:20.549612 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.549576 2573 generic.go:356] "Generic (PLEG): container finished" podID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerID="e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729" exitCode=0 Jul 15 21:34:20.549763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.549622 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" event={"ID":"bf4232ba-9cf9-46e9-9cf7-743515cb7155","Type":"ContainerDied","Data":"e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729"} Jul 15 21:34:20.549763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.549678 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" event={"ID":"bf4232ba-9cf9-46e9-9cf7-743515cb7155","Type":"ContainerDied","Data":"7a1c1c9f3ad914ad067c9fe55e3b9a4f1b5a86b3866fd65566109a734d2c10ab"} Jul 15 21:34:20.549763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.549622 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9" Jul 15 21:34:20.549763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.549708 2573 scope.go:117] "RemoveContainer" containerID="e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729" Jul 15 21:34:20.559349 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.559338 2573 scope.go:117] "RemoveContainer" containerID="b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7" Jul 15 21:34:20.567698 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.567678 2573 scope.go:117] "RemoveContainer" containerID="e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729" Jul 15 21:34:20.567964 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:34:20.567944 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729\": container with ID starting with e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729 not found: ID does not exist" containerID="e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729" Jul 15 21:34:20.568007 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.567973 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729"} err="failed to get container status \"e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729\": rpc error: code = NotFound desc = could not find container \"e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729\": container with ID starting with e34025c3d5545a74b0987656eda7cf7b421fc1b70981f380820b7b8566123729 not found: ID does not exist" Jul 15 21:34:20.568007 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.567990 2573 scope.go:117] "RemoveContainer" containerID="b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7" Jul 15 21:34:20.568242 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:34:20.568221 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7\": container with ID starting with b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7 not found: ID does not exist" containerID="b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7" Jul 15 21:34:20.568345 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.568244 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7"} err="failed to get container status \"b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7\": rpc error: code = NotFound desc = could not find container \"b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7\": container with ID starting with b5c00c0f7cf40ab070c57c68948ac19f08d292f75a3636bb76fa2d692e3c62e7 not found: ID does not exist" Jul 15 21:34:20.634660 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.634635 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-envoy\") pod \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " Jul 15 21:34:20.634798 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.634683 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-token\") pod \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " Jul 15 21:34:20.634798 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.634706 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-certs\") pod \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " Jul 15 21:34:20.634912 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.634797 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istiod-ca-cert\") pod \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " Jul 15 21:34:20.634912 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.634824 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccfq9\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-kube-api-access-ccfq9\") pod \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " Jul 15 21:34:20.634912 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.634852 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-socket\") pod \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " Jul 15 21:34:20.635071 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.634938 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-podinfo\") pod \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " Jul 15 21:34:20.635071 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.634972 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-data\") pod \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " Jul 15 21:34:20.635071 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.635007 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-credential-socket\") pod \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\" (UID: \"bf4232ba-9cf9-46e9-9cf7-743515cb7155\") " Jul 15 21:34:20.635071 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.635031 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bf4232ba-9cf9-46e9-9cf7-743515cb7155" (UID: "bf4232ba-9cf9-46e9-9cf7-743515cb7155"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:20.635316 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.635153 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bf4232ba-9cf9-46e9-9cf7-743515cb7155" (UID: "bf4232ba-9cf9-46e9-9cf7-743515cb7155"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:34:20.635316 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.635200 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bf4232ba-9cf9-46e9-9cf7-743515cb7155" (UID: "bf4232ba-9cf9-46e9-9cf7-743515cb7155"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:20.635465 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.635436 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:34:20.635568 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.635468 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:34:20.635568 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.635482 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:34:20.635568 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.635434 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bf4232ba-9cf9-46e9-9cf7-743515cb7155" (UID: "bf4232ba-9cf9-46e9-9cf7-743515cb7155"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:20.636059 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.636038 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-data" (OuterVolumeSpecName: "istio-data") pod "bf4232ba-9cf9-46e9-9cf7-743515cb7155" (UID: "bf4232ba-9cf9-46e9-9cf7-743515cb7155"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:20.636966 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.636942 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bf4232ba-9cf9-46e9-9cf7-743515cb7155" (UID: "bf4232ba-9cf9-46e9-9cf7-743515cb7155"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:34:20.637056 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.637022 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-token" (OuterVolumeSpecName: "istio-token") pod "bf4232ba-9cf9-46e9-9cf7-743515cb7155" (UID: "bf4232ba-9cf9-46e9-9cf7-743515cb7155"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:34:20.637335 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.637313 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bf4232ba-9cf9-46e9-9cf7-743515cb7155" (UID: "bf4232ba-9cf9-46e9-9cf7-743515cb7155"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:34:20.637335 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.637322 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-kube-api-access-ccfq9" (OuterVolumeSpecName: "kube-api-access-ccfq9") pod "bf4232ba-9cf9-46e9-9cf7-743515cb7155" (UID: "bf4232ba-9cf9-46e9-9cf7-743515cb7155"). InnerVolumeSpecName "kube-api-access-ccfq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:34:20.736061 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.736033 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:34:20.736061 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.736060 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccfq9\" (UniqueName: \"kubernetes.io/projected/bf4232ba-9cf9-46e9-9cf7-743515cb7155-kube-api-access-ccfq9\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:34:20.736061 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.736070 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:34:20.736322 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.736080 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:34:20.736322 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.736089 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:34:20.736322 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.736102 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf4232ba-9cf9-46e9-9cf7-743515cb7155-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:34:20.875841 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.875810 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9"] Jul 15 21:34:20.882868 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:20.882841 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-372e1c00-openshift-default-988dff4ff-tr6n9"] Jul 15 21:34:21.447945 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:34:21.447911 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" path="/var/lib/kubelet/pods/bf4232ba-9cf9-46e9-9cf7-743515cb7155/volumes" Jul 15 21:35:14.274923 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.274889 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98"] Jul 15 21:35:14.275544 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.275524 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" Jul 15 21:35:14.275634 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.275547 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" Jul 15 21:35:14.275634 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.275586 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" Jul 15 21:35:14.275634 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.275595 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" Jul 15 21:35:14.275787 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.275704 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" Jul 15 21:35:14.277960 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.277939 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.280810 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.280790 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-945d24-openshift-default-dockercfg-6mp22\"" Jul 15 21:35:14.290550 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.290523 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98"] Jul 15 21:35:14.332155 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.332122 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-data\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.332320 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.332160 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-envoy\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.332320 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.332255 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-token\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.332468 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.332338 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-certs\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.332468 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.332366 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-socket\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.332468 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.332400 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-credential-socket\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.332595 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.332462 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-podinfo\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.332595 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.332525 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istiod-ca-cert\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.332595 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.332564 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxl7b\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-kube-api-access-pxl7b\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433274 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433236 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-token\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433482 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433334 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-certs\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433482 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433370 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-socket\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433482 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433414 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-credential-socket\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433482 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433439 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-podinfo\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433709 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433595 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istiod-ca-cert\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433709 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433638 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxl7b\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-kube-api-access-pxl7b\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433709 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433691 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-data\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433873 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433729 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-envoy\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433873 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433798 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-certs\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.433979 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433875 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-credential-socket\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.434040 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.433982 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-socket\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.434205 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.434151 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-data\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.434446 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.434426 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istiod-ca-cert\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.435951 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.435933 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-envoy\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.436410 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.436391 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-podinfo\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.441928 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.441906 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-token\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.442128 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.442107 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxl7b\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-kube-api-access-pxl7b\") pod \"e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.588460 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.588365 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:14.736109 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.736071 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98"] Jul 15 21:35:14.737420 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:35:14.737391 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b310eae_e269_46af_b130_9bb0c3a8c5ed.slice/crio-dc630d30444f527f402d7e6f62b7f1068979d7a596c75555079625e56725053b WatchSource:0}: Error finding container dc630d30444f527f402d7e6f62b7f1068979d7a596c75555079625e56725053b: Status 404 returned error can't find the container with id dc630d30444f527f402d7e6f62b7f1068979d7a596c75555079625e56725053b Jul 15 21:35:14.739984 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.739953 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:14.740090 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.740016 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:14.740090 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.740042 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:14.857301 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.857207 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" event={"ID":"7b310eae-e269-46af-b130-9bb0c3a8c5ed","Type":"ContainerStarted","Data":"54fde0f5604318db30a229d1c693834ac2b35c7eb342c0bcbc5f1aef73917e5d"} Jul 15 21:35:14.857301 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.857275 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" event={"ID":"7b310eae-e269-46af-b130-9bb0c3a8c5ed","Type":"ContainerStarted","Data":"dc630d30444f527f402d7e6f62b7f1068979d7a596c75555079625e56725053b"} Jul 15 21:35:14.881203 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:14.881130 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" podStartSLOduration=0.881113048 podStartE2EDuration="881.113048ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:35:14.879627193 +0000 UTC m=+2167.969860388" watchObservedRunningTime="2026-07-15 21:35:14.881113048 +0000 UTC m=+2167.971346226" Jul 15 21:35:15.588599 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:15.588564 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:15.593441 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:15.593412 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:15.863237 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:15.863137 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:15.864028 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:15.864008 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:22.071862 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.071829 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4"] Jul 15 21:35:22.072362 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.072349 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf4232ba-9cf9-46e9-9cf7-743515cb7155" containerName="istio-proxy" Jul 15 21:35:22.074671 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.074656 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.077851 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.077825 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-6874e2-openshift-default-dockercfg-sgk8c\"" Jul 15 21:35:22.087890 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.087859 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4"] Jul 15 21:35:22.107066 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.107035 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-socket\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.107216 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.107067 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-credential-socket\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.107216 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.107090 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-data\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.107216 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.107184 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-certs\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.107322 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.107245 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istiod-ca-cert\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.107322 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.107294 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-token\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.107392 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.107347 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-envoy\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.107426 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.107395 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-podinfo\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.107426 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.107415 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsqlf\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-kube-api-access-rsqlf\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.208851 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.208640 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-envoy\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.209094 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-podinfo\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.209214 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsqlf\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-kube-api-access-rsqlf\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.209303 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-socket\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.209347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-credential-socket\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.209401 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-data\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.209458 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-certs\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.209538 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istiod-ca-cert\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.209613 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-token\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.210664 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-socket\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.210957 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-credential-socket\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.211240 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-data\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.211499 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-certs\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213821 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.211772 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-envoy\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.213821 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.212191 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istiod-ca-cert\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.218198 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.218148 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-podinfo\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.220395 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.220374 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-token\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.220553 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.220527 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsqlf\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-kube-api-access-rsqlf\") pod \"e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.386233 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.386139 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:22.575666 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.575637 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4"] Jul 15 21:35:22.577324 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:35:22.577285 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfa650c9_331e_42b9_ba6c_c932bade9ebe.slice/crio-7089afd9dcf3614f330ebadddd13e449dd961e26cdd420ab0f23a6879145932b WatchSource:0}: Error finding container 7089afd9dcf3614f330ebadddd13e449dd961e26cdd420ab0f23a6879145932b: Status 404 returned error can't find the container with id 7089afd9dcf3614f330ebadddd13e449dd961e26cdd420ab0f23a6879145932b Jul 15 21:35:22.579520 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.579477 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:22.579590 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.579570 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:22.579638 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.579612 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:22.904727 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.904641 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" event={"ID":"cfa650c9-331e-42b9-ba6c-c932bade9ebe","Type":"ContainerStarted","Data":"b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693"} Jul 15 21:35:22.904727 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.904700 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" event={"ID":"cfa650c9-331e-42b9-ba6c-c932bade9ebe","Type":"ContainerStarted","Data":"7089afd9dcf3614f330ebadddd13e449dd961e26cdd420ab0f23a6879145932b"} Jul 15 21:35:22.932718 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:22.929420 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" podStartSLOduration=0.929401301 podStartE2EDuration="929.401301ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:35:22.927531296 +0000 UTC m=+2176.017764475" watchObservedRunningTime="2026-07-15 21:35:22.929401301 +0000 UTC m=+2176.019634481" Jul 15 21:35:23.386330 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:23.386298 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:23.390895 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:23.390875 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:23.911005 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:23.910974 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:23.912025 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:23.912008 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:29.947903 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:29.947820 2573 generic.go:356] "Generic (PLEG): container finished" podID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerID="54fde0f5604318db30a229d1c693834ac2b35c7eb342c0bcbc5f1aef73917e5d" exitCode=0 Jul 15 21:35:29.947903 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:29.947888 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" event={"ID":"7b310eae-e269-46af-b130-9bb0c3a8c5ed","Type":"ContainerDied","Data":"54fde0f5604318db30a229d1c693834ac2b35c7eb342c0bcbc5f1aef73917e5d"} Jul 15 21:35:29.948331 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:29.948296 2573 scope.go:117] "RemoveContainer" containerID="54fde0f5604318db30a229d1c693834ac2b35c7eb342c0bcbc5f1aef73917e5d" Jul 15 21:35:29.949307 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:29.949278 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:29.949365 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:29.949345 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:29.949400 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:29.949376 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:30.863830 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:30.863785 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:30.961542 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:30.961497 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" event={"ID":"7b310eae-e269-46af-b130-9bb0c3a8c5ed","Type":"ContainerStarted","Data":"ff076ebdba5f6b720ef5febc050193da1d4427a0278f55e73e53dd913ab33964"} Jul 15 21:35:31.588766 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:31.588735 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:31.589695 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:31.589675 2573 patch_prober.go:28] interesting pod/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 15 21:35:31.589809 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:31.589712 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 15 21:35:32.588908 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:32.588870 2573 patch_prober.go:28] interesting pod/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 15 21:35:32.589343 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:32.588957 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 15 21:35:33.589612 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:33.589574 2573 patch_prober.go:28] interesting pod/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 15 21:35:33.590024 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:33.589626 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 15 21:35:34.589739 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:34.589688 2573 patch_prober.go:28] interesting pod/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 15 21:35:34.590268 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:34.589756 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 15 21:35:35.594345 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:35.594314 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:35.594802 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:35.594597 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:35.595488 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:35.595466 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:35:38.020043 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:38.020008 2573 generic.go:356] "Generic (PLEG): container finished" podID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerID="b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693" exitCode=0 Jul 15 21:35:38.020476 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:38.020080 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" event={"ID":"cfa650c9-331e-42b9-ba6c-c932bade9ebe","Type":"ContainerDied","Data":"b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693"} Jul 15 21:35:38.020527 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:38.020514 2573 scope.go:117] "RemoveContainer" containerID="b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693" Jul 15 21:35:38.021557 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:38.021531 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:38.021629 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:38.021602 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:38.021666 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:38.021641 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:35:38.911585 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:38.911551 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:39.029396 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:39.029362 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" event={"ID":"cfa650c9-331e-42b9-ba6c-c932bade9ebe","Type":"ContainerStarted","Data":"b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503"} Jul 15 21:35:39.386469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:39.386443 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:39.387502 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:39.387481 2573 patch_prober.go:28] interesting pod/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 15 21:35:39.387574 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:39.387528 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 15 21:35:40.387469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:40.387430 2573 patch_prober.go:28] interesting pod/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 15 21:35:40.387871 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:40.387490 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 15 21:35:41.386964 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:41.386929 2573 patch_prober.go:28] interesting pod/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 15 21:35:41.387142 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:41.386998 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 15 21:35:42.387432 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:42.387395 2573 patch_prober.go:28] interesting pod/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 15 21:35:42.387821 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:42.387458 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 15 21:35:43.390628 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:43.390598 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:43.391002 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:43.390982 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:35:43.391736 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:35:43.391717 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:36:52.305016 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:52.304941 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98"] Jul 15 21:36:52.305433 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:52.305160 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" containerID="cri-o://ff076ebdba5f6b720ef5febc050193da1d4427a0278f55e73e53dd913ab33964" gracePeriod=30 Jul 15 21:36:57.521717 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.521683 2573 generic.go:356] "Generic (PLEG): container finished" podID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerID="ff076ebdba5f6b720ef5febc050193da1d4427a0278f55e73e53dd913ab33964" exitCode=0 Jul 15 21:36:57.522137 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.521765 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" event={"ID":"7b310eae-e269-46af-b130-9bb0c3a8c5ed","Type":"ContainerDied","Data":"ff076ebdba5f6b720ef5febc050193da1d4427a0278f55e73e53dd913ab33964"} Jul 15 21:36:57.522137 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.521843 2573 scope.go:117] "RemoveContainer" containerID="54fde0f5604318db30a229d1c693834ac2b35c7eb342c0bcbc5f1aef73917e5d" Jul 15 21:36:57.558641 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.558618 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:36:57.608899 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.608815 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istiod-ca-cert\") pod \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " Jul 15 21:36:57.608899 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.608886 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-socket\") pod \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " Jul 15 21:36:57.609116 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.608918 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-certs\") pod \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " Jul 15 21:36:57.609116 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609008 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxl7b\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-kube-api-access-pxl7b\") pod \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " Jul 15 21:36:57.609116 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609055 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-credential-socket\") pod \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " Jul 15 21:36:57.609116 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609099 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-podinfo\") pod \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " Jul 15 21:36:57.609363 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609130 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-envoy\") pod \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " Jul 15 21:36:57.609363 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609180 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-data\") pod \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " Jul 15 21:36:57.609363 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609211 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-token\") pod \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\" (UID: \"7b310eae-e269-46af-b130-9bb0c3a8c5ed\") " Jul 15 21:36:57.609363 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609241 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7b310eae-e269-46af-b130-9bb0c3a8c5ed" (UID: "7b310eae-e269-46af-b130-9bb0c3a8c5ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:36:57.609363 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609256 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7b310eae-e269-46af-b130-9bb0c3a8c5ed" (UID: "7b310eae-e269-46af-b130-9bb0c3a8c5ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:36:57.609633 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609393 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7b310eae-e269-46af-b130-9bb0c3a8c5ed" (UID: "7b310eae-e269-46af-b130-9bb0c3a8c5ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:36:57.609633 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609587 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:36:57.609633 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609608 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:36:57.609633 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609622 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:36:57.609835 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.609639 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7b310eae-e269-46af-b130-9bb0c3a8c5ed" (UID: "7b310eae-e269-46af-b130-9bb0c3a8c5ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:36:57.610057 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.610031 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "7b310eae-e269-46af-b130-9bb0c3a8c5ed" (UID: "7b310eae-e269-46af-b130-9bb0c3a8c5ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:36:57.611296 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.611262 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-kube-api-access-pxl7b" (OuterVolumeSpecName: "kube-api-access-pxl7b") pod "7b310eae-e269-46af-b130-9bb0c3a8c5ed" (UID: "7b310eae-e269-46af-b130-9bb0c3a8c5ed"). InnerVolumeSpecName "kube-api-access-pxl7b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:36:57.611500 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.611478 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7b310eae-e269-46af-b130-9bb0c3a8c5ed" (UID: "7b310eae-e269-46af-b130-9bb0c3a8c5ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:36:57.611600 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.611493 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7b310eae-e269-46af-b130-9bb0c3a8c5ed" (UID: "7b310eae-e269-46af-b130-9bb0c3a8c5ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:36:57.611807 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.611791 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "7b310eae-e269-46af-b130-9bb0c3a8c5ed" (UID: "7b310eae-e269-46af-b130-9bb0c3a8c5ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:36:57.710860 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.710829 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxl7b\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-kube-api-access-pxl7b\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:36:57.710860 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.710853 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:36:57.710860 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.710862 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:36:57.710860 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.710870 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:36:57.711117 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.710879 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:36:57.711117 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:57.710886 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b310eae-e269-46af-b130-9bb0c3a8c5ed-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:36:58.528786 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:58.528759 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" Jul 15 21:36:58.529237 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:58.528747 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98" event={"ID":"7b310eae-e269-46af-b130-9bb0c3a8c5ed","Type":"ContainerDied","Data":"dc630d30444f527f402d7e6f62b7f1068979d7a596c75555079625e56725053b"} Jul 15 21:36:58.529237 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:58.528891 2573 scope.go:117] "RemoveContainer" containerID="ff076ebdba5f6b720ef5febc050193da1d4427a0278f55e73e53dd913ab33964" Jul 15 21:36:58.555452 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:58.555420 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98"] Jul 15 21:36:58.560246 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:58.560222 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-945d24-openshift-default-9bc99794f-mqm98"] Jul 15 21:36:59.447965 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:36:59.447933 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" path="/var/lib/kubelet/pods/7b310eae-e269-46af-b130-9bb0c3a8c5ed/volumes" Jul 15 21:37:14.912132 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:14.912099 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4"] Jul 15 21:37:14.912599 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:14.912326 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" containerID="cri-o://b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503" gracePeriod=30 Jul 15 21:37:20.156226 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.156202 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:37:20.326050 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326018 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-socket\") pod \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " Jul 15 21:37:20.326290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326060 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-podinfo\") pod \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " Jul 15 21:37:20.326290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326109 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-credential-socket\") pod \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " Jul 15 21:37:20.326290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326139 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-data\") pod \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " Jul 15 21:37:20.326475 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326387 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cfa650c9-331e-42b9-ba6c-c932bade9ebe" (UID: "cfa650c9-331e-42b9-ba6c-c932bade9ebe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:20.326475 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326452 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-envoy\") pod \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " Jul 15 21:37:20.326588 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326498 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-certs\") pod \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " Jul 15 21:37:20.326588 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326547 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-token\") pod \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " Jul 15 21:37:20.326698 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326589 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istiod-ca-cert\") pod \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " Jul 15 21:37:20.326698 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326597 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cfa650c9-331e-42b9-ba6c-c932bade9ebe" (UID: "cfa650c9-331e-42b9-ba6c-c932bade9ebe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:20.326698 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326663 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsqlf\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-kube-api-access-rsqlf\") pod \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\" (UID: \"cfa650c9-331e-42b9-ba6c-c932bade9ebe\") " Jul 15 21:37:20.326869 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.326838 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cfa650c9-331e-42b9-ba6c-c932bade9ebe" (UID: "cfa650c9-331e-42b9-ba6c-c932bade9ebe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:20.327063 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.327043 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:20.327128 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.327071 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:20.327128 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.327086 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:20.327402 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.327378 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-data" (OuterVolumeSpecName: "istio-data") pod "cfa650c9-331e-42b9-ba6c-c932bade9ebe" (UID: "cfa650c9-331e-42b9-ba6c-c932bade9ebe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:20.327553 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.327530 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cfa650c9-331e-42b9-ba6c-c932bade9ebe" (UID: "cfa650c9-331e-42b9-ba6c-c932bade9ebe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:37:20.328956 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.328918 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cfa650c9-331e-42b9-ba6c-c932bade9ebe" (UID: "cfa650c9-331e-42b9-ba6c-c932bade9ebe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:37:20.329199 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.329154 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-kube-api-access-rsqlf" (OuterVolumeSpecName: "kube-api-access-rsqlf") pod "cfa650c9-331e-42b9-ba6c-c932bade9ebe" (UID: "cfa650c9-331e-42b9-ba6c-c932bade9ebe"). InnerVolumeSpecName "kube-api-access-rsqlf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:37:20.329917 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.329893 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cfa650c9-331e-42b9-ba6c-c932bade9ebe" (UID: "cfa650c9-331e-42b9-ba6c-c932bade9ebe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:37:20.330009 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.329945 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-token" (OuterVolumeSpecName: "istio-token") pod "cfa650c9-331e-42b9-ba6c-c932bade9ebe" (UID: "cfa650c9-331e-42b9-ba6c-c932bade9ebe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:37:20.428024 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.427996 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsqlf\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-kube-api-access-rsqlf\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:20.428024 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.428024 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:20.428257 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.428037 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:20.428257 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.428049 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:20.428257 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.428061 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:20.428257 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.428071 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa650c9-331e-42b9-ba6c-c932bade9ebe-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:20.653930 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.653847 2573 generic.go:356] "Generic (PLEG): container finished" podID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerID="b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503" exitCode=0 Jul 15 21:37:20.653930 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.653900 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" Jul 15 21:37:20.654106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.653924 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" event={"ID":"cfa650c9-331e-42b9-ba6c-c932bade9ebe","Type":"ContainerDied","Data":"b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503"} Jul 15 21:37:20.654106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.653981 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4" event={"ID":"cfa650c9-331e-42b9-ba6c-c932bade9ebe","Type":"ContainerDied","Data":"7089afd9dcf3614f330ebadddd13e449dd961e26cdd420ab0f23a6879145932b"} Jul 15 21:37:20.654106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.654002 2573 scope.go:117] "RemoveContainer" containerID="b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503" Jul 15 21:37:20.663768 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.663750 2573 scope.go:117] "RemoveContainer" containerID="b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693" Jul 15 21:37:20.672859 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.672838 2573 scope.go:117] "RemoveContainer" containerID="b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503" Jul 15 21:37:20.673096 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:37:20.673077 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503\": container with ID starting with b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503 not found: ID does not exist" containerID="b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503" Jul 15 21:37:20.673145 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.673104 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503"} err="failed to get container status \"b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503\": rpc error: code = NotFound desc = could not find container \"b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503\": container with ID starting with b97d355422cacb52f7dd56d1e02a8dc5ee3f5fc88116e019c58087733da56503 not found: ID does not exist" Jul 15 21:37:20.673145 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.673120 2573 scope.go:117] "RemoveContainer" containerID="b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693" Jul 15 21:37:20.673390 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:37:20.673370 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693\": container with ID starting with b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693 not found: ID does not exist" containerID="b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693" Jul 15 21:37:20.673438 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.673396 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693"} err="failed to get container status \"b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693\": rpc error: code = NotFound desc = could not find container \"b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693\": container with ID starting with b1429958bee98b4283c0ad62416060dc72e22b986158fe1cb1ebd1d8cb841693 not found: ID does not exist" Jul 15 21:37:20.678809 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.678788 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4"] Jul 15 21:37:20.683655 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:20.683636 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-6874e2-openshift-default-5b57874c7f-4snp4"] Jul 15 21:37:21.447688 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:21.447646 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" path="/var/lib/kubelet/pods/cfa650c9-331e-42b9-ba6c-c932bade9ebe/volumes" Jul 15 21:37:26.983946 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.983915 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5"] Jul 15 21:37:26.984367 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984354 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" Jul 15 21:37:26.984367 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984369 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" Jul 15 21:37:26.984448 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984378 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" Jul 15 21:37:26.984448 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984384 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" Jul 15 21:37:26.984448 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984397 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" Jul 15 21:37:26.984448 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984402 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" Jul 15 21:37:26.984448 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984418 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" Jul 15 21:37:26.984448 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984423 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" Jul 15 21:37:26.984636 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984489 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" Jul 15 21:37:26.984636 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984497 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" Jul 15 21:37:26.984636 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.984502 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b310eae-e269-46af-b130-9bb0c3a8c5ed" containerName="istio-proxy" Jul 15 21:37:26.987697 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.987679 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:26.990965 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.990946 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-351fa3-openshift-default-dockercfg-xzppd\"" Jul 15 21:37:26.998882 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.998861 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-credential-socket\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:26.998983 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.998896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-certs\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:26.998983 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.998927 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-socket\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:26.999100 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.998982 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istiod-ca-cert\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:26.999100 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.999017 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-envoy\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:26.999100 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.999042 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-token\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:26.999100 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.999071 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-podinfo\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:26.999299 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.999108 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-data\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:26.999299 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:26.999231 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8npj\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-kube-api-access-w8npj\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.003673 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.003649 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5"] Jul 15 21:37:27.100360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100326 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-credential-socket\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100371 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-certs\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100398 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-socket\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100451 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istiod-ca-cert\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100474 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-envoy\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100499 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-token\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-podinfo\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100571 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-data\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100650 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100643 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8npj\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-kube-api-access-w8npj\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100994 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100791 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-certs\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.100994 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.100870 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-credential-socket\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.101070 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.101031 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-data\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.101154 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.101131 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-socket\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.101229 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.101212 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istiod-ca-cert\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.102865 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.102837 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-envoy\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.103059 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.103043 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-podinfo\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.110109 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.110087 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-token\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.110396 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.110376 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8npj\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-kube-api-access-w8npj\") pod \"e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.298709 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.298672 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:27.448902 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.448839 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5"] Jul 15 21:37:27.451893 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:37:27.451869 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70227c9_e036_44fa_a0c5_f6b2e6f032e0.slice/crio-07de67179691ae5854f11a51552621b83fa302576b4ae3868e604a91950beb98 WatchSource:0}: Error finding container 07de67179691ae5854f11a51552621b83fa302576b4ae3868e604a91950beb98: Status 404 returned error can't find the container with id 07de67179691ae5854f11a51552621b83fa302576b4ae3868e604a91950beb98 Jul 15 21:37:27.454011 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.453984 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:37:27.454072 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.454044 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:37:27.454107 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.454070 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:37:27.702677 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.702581 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" event={"ID":"e70227c9-e036-44fa-a0c5-f6b2e6f032e0","Type":"ContainerStarted","Data":"ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38"} Jul 15 21:37:27.702677 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.702670 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" event={"ID":"e70227c9-e036-44fa-a0c5-f6b2e6f032e0","Type":"ContainerStarted","Data":"07de67179691ae5854f11a51552621b83fa302576b4ae3868e604a91950beb98"} Jul 15 21:37:27.746246 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:27.746194 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" podStartSLOduration=1.7461801019999998 podStartE2EDuration="1.746180102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:37:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:37:27.74225222 +0000 UTC m=+2300.832485397" watchObservedRunningTime="2026-07-15 21:37:27.746180102 +0000 UTC m=+2300.836413271" Jul 15 21:37:28.298812 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:28.298781 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:28.303660 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:28.303639 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:28.708828 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:28.708749 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:28.709566 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:28.709542 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:41.799324 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:41.799288 2573 generic.go:356] "Generic (PLEG): container finished" podID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerID="ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38" exitCode=0 Jul 15 21:37:41.799704 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:41.799366 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" event={"ID":"e70227c9-e036-44fa-a0c5-f6b2e6f032e0","Type":"ContainerDied","Data":"ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38"} Jul 15 21:37:41.799814 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:41.799799 2573 scope.go:117] "RemoveContainer" containerID="ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38" Jul 15 21:37:41.800843 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:41.800819 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:37:41.800905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:41.800883 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:37:41.800940 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:41.800924 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 21:37:42.369421 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.369385 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88"] Jul 15 21:37:42.369873 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.369859 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfa650c9-331e-42b9-ba6c-c932bade9ebe" containerName="istio-proxy" Jul 15 21:37:42.372522 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.372506 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" Jul 15 21:37:42.381324 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.381293 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kpm6w\"" Jul 15 21:37:42.385507 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.385483 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88"] Jul 15 21:37:42.455652 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.455611 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88\" (UID: \"7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" Jul 15 21:37:42.557396 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.557346 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88\" (UID: \"7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" Jul 15 21:37:42.558192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.558137 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88\" (UID: \"7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" Jul 15 21:37:42.694368 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.694282 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" Jul 15 21:37:42.810374 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.809793 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" event={"ID":"e70227c9-e036-44fa-a0c5-f6b2e6f032e0","Type":"ContainerStarted","Data":"2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5"} Jul 15 21:37:42.886440 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:42.886410 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88"] Jul 15 21:37:42.891741 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:37:42.891697 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea91beb_6cb5_44bc_9f74_9e17c8b1ed22.slice/crio-d9df244f50d344c1861aa7432bdb8183cce3f985873032a2f3b1095fb1501556 WatchSource:0}: Error finding container d9df244f50d344c1861aa7432bdb8183cce3f985873032a2f3b1095fb1501556: Status 404 returned error can't find the container with id d9df244f50d344c1861aa7432bdb8183cce3f985873032a2f3b1095fb1501556 Jul 15 21:37:43.299212 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:43.299152 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:43.300445 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:43.300428 2573 patch_prober.go:28] interesting pod/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 15 21:37:43.300502 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:43.300469 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 15 21:37:43.708842 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:43.708758 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:43.818183 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:43.818143 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" event={"ID":"7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22","Type":"ContainerStarted","Data":"d98a9e7e3880b57ff1fd666ea2ae1ace6a728450692f4af9d8eaa7b9f4eaab2a"} Jul 15 21:37:43.818651 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:43.818224 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" event={"ID":"7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22","Type":"ContainerStarted","Data":"d9df244f50d344c1861aa7432bdb8183cce3f985873032a2f3b1095fb1501556"} Jul 15 21:37:43.841213 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:43.841130 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" podStartSLOduration=1.841110169 podStartE2EDuration="1.841110169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:37:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:37:43.835921857 +0000 UTC m=+2316.926155061" watchObservedRunningTime="2026-07-15 21:37:43.841110169 +0000 UTC m=+2316.931343348" Jul 15 21:37:44.299064 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:44.299026 2573 patch_prober.go:28] interesting pod/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 15 21:37:44.299264 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:44.299083 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 15 21:37:44.826141 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:44.826106 2573 generic.go:356] "Generic (PLEG): container finished" podID="7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22" containerID="d98a9e7e3880b57ff1fd666ea2ae1ace6a728450692f4af9d8eaa7b9f4eaab2a" exitCode=0 Jul 15 21:37:44.826547 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:44.826210 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" event={"ID":"7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22","Type":"ContainerDied","Data":"d98a9e7e3880b57ff1fd666ea2ae1ace6a728450692f4af9d8eaa7b9f4eaab2a"} Jul 15 21:37:45.299406 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:45.299370 2573 patch_prober.go:28] interesting pod/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 15 21:37:45.299567 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:45.299432 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 15 21:37:46.003735 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:46.003707 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" Jul 15 21:37:46.093778 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:46.093699 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22-maas-api-service-ca\") pod \"7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22\" (UID: \"7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22\") " Jul 15 21:37:46.094019 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:46.093998 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22" (UID: "7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:37:46.195089 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:46.195039 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22-maas-api-service-ca\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:37:46.300049 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:46.300015 2573 patch_prober.go:28] interesting pod/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 15 21:37:46.300233 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:46.300072 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 15 21:37:46.842819 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:46.842785 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" Jul 15 21:37:46.843005 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:46.842827 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88" event={"ID":"7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22","Type":"ContainerDied","Data":"d9df244f50d344c1861aa7432bdb8183cce3f985873032a2f3b1095fb1501556"} Jul 15 21:37:46.843005 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:46.842882 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9df244f50d344c1861aa7432bdb8183cce3f985873032a2f3b1095fb1501556" Jul 15 21:37:47.304232 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:47.304199 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:37:47.305270 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:37:47.305251 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:38:09.915510 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:09.915467 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk"] Jul 15 21:38:09.915905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:09.915888 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22" containerName="revoke-keys" Jul 15 21:38:09.915905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:09.915899 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22" containerName="revoke-keys" Jul 15 21:38:09.915992 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:09.915981 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22" containerName="revoke-keys" Jul 15 21:38:09.918882 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:09.918855 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" Jul 15 21:38:09.921584 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:09.921561 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kpm6w\"" Jul 15 21:38:09.928438 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:09.928415 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk"] Jul 15 21:38:10.029674 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:10.029633 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bafe0d-99ba-4ce8-94fd-0846dbcdd18b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk\" (UID: \"32bafe0d-99ba-4ce8-94fd-0846dbcdd18b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" Jul 15 21:38:10.130763 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:10.130715 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bafe0d-99ba-4ce8-94fd-0846dbcdd18b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk\" (UID: \"32bafe0d-99ba-4ce8-94fd-0846dbcdd18b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" Jul 15 21:38:10.131431 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:10.131405 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bafe0d-99ba-4ce8-94fd-0846dbcdd18b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk\" (UID: \"32bafe0d-99ba-4ce8-94fd-0846dbcdd18b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" Jul 15 21:38:10.231340 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:10.231239 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" Jul 15 21:38:10.377439 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:10.377410 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk"] Jul 15 21:38:10.379452 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:38:10.379421 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32bafe0d_99ba_4ce8_94fd_0846dbcdd18b.slice/crio-c17f6c9046cac85e4d7b814636939eb4b3f028cf2d83d892010aea793ea1bb0e WatchSource:0}: Error finding container c17f6c9046cac85e4d7b814636939eb4b3f028cf2d83d892010aea793ea1bb0e: Status 404 returned error can't find the container with id c17f6c9046cac85e4d7b814636939eb4b3f028cf2d83d892010aea793ea1bb0e Jul 15 21:38:10.986953 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:10.986861 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" event={"ID":"32bafe0d-99ba-4ce8-94fd-0846dbcdd18b","Type":"ContainerStarted","Data":"cd001d8e83a2de0b26eb6f84edfa2e1393f85062c9c731cfd5c7c131b427e272"} Jul 15 21:38:10.986953 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:10.986921 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" event={"ID":"32bafe0d-99ba-4ce8-94fd-0846dbcdd18b","Type":"ContainerStarted","Data":"c17f6c9046cac85e4d7b814636939eb4b3f028cf2d83d892010aea793ea1bb0e"} Jul 15 21:38:11.010222 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:11.010156 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" podStartSLOduration=2.01014097 podStartE2EDuration="2.01014097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:38:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:38:11.007747383 +0000 UTC m=+2344.097980616" watchObservedRunningTime="2026-07-15 21:38:11.01014097 +0000 UTC m=+2344.100374148" Jul 15 21:38:11.993953 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:11.993913 2573 generic.go:356] "Generic (PLEG): container finished" podID="32bafe0d-99ba-4ce8-94fd-0846dbcdd18b" containerID="cd001d8e83a2de0b26eb6f84edfa2e1393f85062c9c731cfd5c7c131b427e272" exitCode=0 Jul 15 21:38:11.994354 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:11.993997 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" event={"ID":"32bafe0d-99ba-4ce8-94fd-0846dbcdd18b","Type":"ContainerDied","Data":"cd001d8e83a2de0b26eb6f84edfa2e1393f85062c9c731cfd5c7c131b427e272"} Jul 15 21:38:13.141627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:13.141603 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" Jul 15 21:38:13.260538 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:13.260510 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bafe0d-99ba-4ce8-94fd-0846dbcdd18b-maas-api-service-ca\") pod \"32bafe0d-99ba-4ce8-94fd-0846dbcdd18b\" (UID: \"32bafe0d-99ba-4ce8-94fd-0846dbcdd18b\") " Jul 15 21:38:13.260886 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:13.260861 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32bafe0d-99ba-4ce8-94fd-0846dbcdd18b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "32bafe0d-99ba-4ce8-94fd-0846dbcdd18b" (UID: "32bafe0d-99ba-4ce8-94fd-0846dbcdd18b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:38:13.361798 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:13.361763 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32bafe0d-99ba-4ce8-94fd-0846dbcdd18b-maas-api-service-ca\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:14.006575 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:14.006539 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" event={"ID":"32bafe0d-99ba-4ce8-94fd-0846dbcdd18b","Type":"ContainerDied","Data":"c17f6c9046cac85e4d7b814636939eb4b3f028cf2d83d892010aea793ea1bb0e"} Jul 15 21:38:14.006575 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:14.006559 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk" Jul 15 21:38:14.006780 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:14.006585 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c17f6c9046cac85e4d7b814636939eb4b3f028cf2d83d892010aea793ea1bb0e" Jul 15 21:38:37.150752 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:37.150712 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5"] Jul 15 21:38:37.151715 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:37.151052 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" containerID="cri-o://2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5" gracePeriod=30 Jul 15 21:38:42.413826 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.413759 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:38:42.444112 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444066 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-envoy\") pod \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " Jul 15 21:38:42.444301 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444158 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-credential-socket\") pod \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " Jul 15 21:38:42.444301 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444206 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-token\") pod \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " Jul 15 21:38:42.444301 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444258 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-podinfo\") pod \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " Jul 15 21:38:42.444497 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444324 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8npj\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-kube-api-access-w8npj\") pod \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " Jul 15 21:38:42.444497 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444361 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istiod-ca-cert\") pod \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " Jul 15 21:38:42.444497 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444402 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-socket\") pod \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " Jul 15 21:38:42.444497 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444439 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-data\") pod \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " Jul 15 21:38:42.444789 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444495 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e70227c9-e036-44fa-a0c5-f6b2e6f032e0" (UID: "e70227c9-e036-44fa-a0c5-f6b2e6f032e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:38:42.444789 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444519 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-certs\") pod \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\" (UID: \"e70227c9-e036-44fa-a0c5-f6b2e6f032e0\") " Jul 15 21:38:42.444905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444796 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-credential-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:42.444905 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.444800 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e70227c9-e036-44fa-a0c5-f6b2e6f032e0" (UID: "e70227c9-e036-44fa-a0c5-f6b2e6f032e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:38:42.445366 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.445343 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e70227c9-e036-44fa-a0c5-f6b2e6f032e0" (UID: "e70227c9-e036-44fa-a0c5-f6b2e6f032e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:38:42.446599 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.446571 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e70227c9-e036-44fa-a0c5-f6b2e6f032e0" (UID: "e70227c9-e036-44fa-a0c5-f6b2e6f032e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:38:42.446889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.446855 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e70227c9-e036-44fa-a0c5-f6b2e6f032e0" (UID: "e70227c9-e036-44fa-a0c5-f6b2e6f032e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:38:42.447088 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.447045 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-data" (OuterVolumeSpecName: "istio-data") pod "e70227c9-e036-44fa-a0c5-f6b2e6f032e0" (UID: "e70227c9-e036-44fa-a0c5-f6b2e6f032e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 21:38:42.447200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.447088 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-token" (OuterVolumeSpecName: "istio-token") pod "e70227c9-e036-44fa-a0c5-f6b2e6f032e0" (UID: "e70227c9-e036-44fa-a0c5-f6b2e6f032e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:38:42.447200 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.447123 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-kube-api-access-w8npj" (OuterVolumeSpecName: "kube-api-access-w8npj") pod "e70227c9-e036-44fa-a0c5-f6b2e6f032e0" (UID: "e70227c9-e036-44fa-a0c5-f6b2e6f032e0"). InnerVolumeSpecName "kube-api-access-w8npj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:38:42.447480 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.447442 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e70227c9-e036-44fa-a0c5-f6b2e6f032e0" (UID: "e70227c9-e036-44fa-a0c5-f6b2e6f032e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 21:38:42.545759 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.545723 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-certs\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:42.545759 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.545749 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-envoy\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:42.545759 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.545757 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-token\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:42.545759 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.545765 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-podinfo\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:42.546226 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.545774 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8npj\" (UniqueName: \"kubernetes.io/projected/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-kube-api-access-w8npj\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:42.546226 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.545782 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istiod-ca-cert\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:42.546226 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.545790 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-workload-socket\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:42.546226 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:42.545798 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70227c9-e036-44fa-a0c5-f6b2e6f032e0-istio-data\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:38:43.194841 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.194807 2573 generic.go:356] "Generic (PLEG): container finished" podID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerID="2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5" exitCode=0 Jul 15 21:38:43.195003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.194856 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" Jul 15 21:38:43.195003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.194893 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" event={"ID":"e70227c9-e036-44fa-a0c5-f6b2e6f032e0","Type":"ContainerDied","Data":"2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5"} Jul 15 21:38:43.195003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.194952 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5" event={"ID":"e70227c9-e036-44fa-a0c5-f6b2e6f032e0","Type":"ContainerDied","Data":"07de67179691ae5854f11a51552621b83fa302576b4ae3868e604a91950beb98"} Jul 15 21:38:43.195003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.194975 2573 scope.go:117] "RemoveContainer" containerID="2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5" Jul 15 21:38:43.207760 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.207743 2573 scope.go:117] "RemoveContainer" containerID="ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38" Jul 15 21:38:43.216160 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.216141 2573 scope.go:117] "RemoveContainer" containerID="2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5" Jul 15 21:38:43.216464 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:38:43.216444 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5\": container with ID starting with 2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5 not found: ID does not exist" containerID="2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5" Jul 15 21:38:43.216520 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.216473 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5"} err="failed to get container status \"2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5\": rpc error: code = NotFound desc = could not find container \"2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5\": container with ID starting with 2644342fa106a2c5b39c32f732bbaf663de7ef9874e34c3f429fd874c07567f5 not found: ID does not exist" Jul 15 21:38:43.216520 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.216490 2573 scope.go:117] "RemoveContainer" containerID="ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38" Jul 15 21:38:43.216737 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:38:43.216712 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38\": container with ID starting with ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38 not found: ID does not exist" containerID="ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38" Jul 15 21:38:43.216837 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.216746 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38"} err="failed to get container status \"ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38\": rpc error: code = NotFound desc = could not find container \"ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38\": container with ID starting with ad4917ddb7286598872c01ad87bc4a7728c594d9e4b6c53abe4aab652162fe38 not found: ID does not exist" Jul 15 21:38:43.220604 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.220584 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5"] Jul 15 21:38:43.228700 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.228660 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-351fa3-openshift-default-7dc5bd477f-hsfn5"] Jul 15 21:38:43.447975 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:43.447893 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" path="/var/lib/kubelet/pods/e70227c9-e036-44fa-a0c5-f6b2e6f032e0/volumes" Jul 15 21:38:49.287463 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.287425 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5"] Jul 15 21:38:49.288108 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.288091 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" Jul 15 21:38:49.288157 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.288112 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" Jul 15 21:38:49.288157 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.288132 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" Jul 15 21:38:49.288157 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.288141 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" Jul 15 21:38:49.288295 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.288164 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32bafe0d-99ba-4ce8-94fd-0846dbcdd18b" containerName="revoke-keys" Jul 15 21:38:49.288295 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.288189 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="32bafe0d-99ba-4ce8-94fd-0846dbcdd18b" containerName="revoke-keys" Jul 15 21:38:49.288358 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.288301 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" Jul 15 21:38:49.288358 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.288319 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70227c9-e036-44fa-a0c5-f6b2e6f032e0" containerName="istio-proxy" Jul 15 21:38:49.288358 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.288329 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="32bafe0d-99ba-4ce8-94fd-0846dbcdd18b" containerName="revoke-keys" Jul 15 21:38:49.293414 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.293391 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:49.297613 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.297587 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-feaa76-serving-cert\"" Jul 15 21:38:49.305725 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.305701 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5"] Jul 15 21:38:49.410233 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.410194 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/100a016b-8477-4085-a424-980b2b3a5e9b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:49.410408 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.410239 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/100a016b-8477-4085-a424-980b2b3a5e9b-maas-api-tls\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:49.410408 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.410354 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrncr\" (UniqueName: \"kubernetes.io/projected/100a016b-8477-4085-a424-980b2b3a5e9b-kube-api-access-vrncr\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:49.511459 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.511429 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrncr\" (UniqueName: \"kubernetes.io/projected/100a016b-8477-4085-a424-980b2b3a5e9b-kube-api-access-vrncr\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:49.511685 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.511586 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/100a016b-8477-4085-a424-980b2b3a5e9b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:49.511685 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.511647 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/100a016b-8477-4085-a424-980b2b3a5e9b-maas-api-tls\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:49.511808 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:38:49.511752 2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-serving-cert: secret "maas-api-e2e-shared-b-feaa76-serving-cert" not found Jul 15 21:38:49.511849 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:38:49.511830 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/100a016b-8477-4085-a424-980b2b3a5e9b-maas-api-tls podName:100a016b-8477-4085-a424-980b2b3a5e9b nodeName:}" failed. No retries permitted until 2026-07-15 21:38:50.011808992 +0000 UTC m=+2383.102042175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/100a016b-8477-4085-a424-980b2b3a5e9b-maas-api-tls") pod "maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" (UID: "100a016b-8477-4085-a424-980b2b3a5e9b") : secret "maas-api-e2e-shared-b-feaa76-serving-cert" not found Jul 15 21:38:49.512187 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.512151 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/100a016b-8477-4085-a424-980b2b3a5e9b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:49.521803 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:49.521777 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrncr\" (UniqueName: \"kubernetes.io/projected/100a016b-8477-4085-a424-980b2b3a5e9b-kube-api-access-vrncr\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:50.016282 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:50.016232 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/100a016b-8477-4085-a424-980b2b3a5e9b-maas-api-tls\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:50.018723 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:50.018685 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/100a016b-8477-4085-a424-980b2b3a5e9b-maas-api-tls\") pod \"maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:50.207678 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:50.207639 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:50.353531 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:50.353506 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5"] Jul 15 21:38:50.355236 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:38:50.355158 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod100a016b_8477_4085_a424_980b2b3a5e9b.slice/crio-5fd096908a592740ae7c98be6d56d42fd1e0eb712984508bb189996274e8d7d9 WatchSource:0}: Error finding container 5fd096908a592740ae7c98be6d56d42fd1e0eb712984508bb189996274e8d7d9: Status 404 returned error can't find the container with id 5fd096908a592740ae7c98be6d56d42fd1e0eb712984508bb189996274e8d7d9 Jul 15 21:38:50.356663 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:50.356644 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:38:51.249584 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:51.249493 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" event={"ID":"100a016b-8477-4085-a424-980b2b3a5e9b","Type":"ContainerStarted","Data":"84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85"} Jul 15 21:38:51.249584 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:51.249553 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" event={"ID":"100a016b-8477-4085-a424-980b2b3a5e9b","Type":"ContainerStarted","Data":"5fd096908a592740ae7c98be6d56d42fd1e0eb712984508bb189996274e8d7d9"} Jul 15 21:38:51.249798 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:51.249688 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:38:51.271552 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:51.271511 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" podStartSLOduration=1.85780594 podStartE2EDuration="2.27149756s" totalImagesPullingTime="413.69162ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:38:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:38:51.26972613 +0000 UTC m=+2384.359959322" watchObservedRunningTime="2026-07-15 21:38:51.27149756 +0000 UTC m=+2384.361730738" Jul 15 21:38:57.260359 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:38:57.260332 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:39:07.618586 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:39:07.618557 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:39:07.622255 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:39:07.622232 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:39:07.622255 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:39:07.622247 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:39:07.626205 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:39:07.626186 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:44:07.660837 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:07.660799 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:44:07.664616 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:07.664592 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:44:07.667406 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:07.667383 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:44:07.670944 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:07.670926 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2zh8_cb55e4a2-16a3-4bcd-8ab1-4bf14d79d72d/ovn-acl-logging/0.log" Jul 15 21:44:34.202303 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.202265 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7272-140051140803504"] Jul 15 21:44:34.205829 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.205812 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051140803504" Jul 15 21:44:34.208745 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.208724 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-p4dn5\"" Jul 15 21:44:34.216522 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.216500 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051140803504"] Jul 15 21:44:34.317487 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.317451 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-479sf\" (UniqueName: \"kubernetes.io/projected/c69ecc0a-82dd-4b9a-a242-05467c3a2e66-kube-api-access-479sf\") pod \"test-curl-7272-140051140803504\" (UID: \"c69ecc0a-82dd-4b9a-a242-05467c3a2e66\") " pod="opendatahub/test-curl-7272-140051140803504" Jul 15 21:44:34.418666 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.418628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-479sf\" (UniqueName: \"kubernetes.io/projected/c69ecc0a-82dd-4b9a-a242-05467c3a2e66-kube-api-access-479sf\") pod \"test-curl-7272-140051140803504\" (UID: \"c69ecc0a-82dd-4b9a-a242-05467c3a2e66\") " pod="opendatahub/test-curl-7272-140051140803504" Jul 15 21:44:34.428855 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.428824 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-479sf\" (UniqueName: \"kubernetes.io/projected/c69ecc0a-82dd-4b9a-a242-05467c3a2e66-kube-api-access-479sf\") pod \"test-curl-7272-140051140803504\" (UID: \"c69ecc0a-82dd-4b9a-a242-05467c3a2e66\") " pod="opendatahub/test-curl-7272-140051140803504" Jul 15 21:44:34.515122 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.515089 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051140803504" Jul 15 21:44:34.655805 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.655780 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051140803504"] Jul 15 21:44:34.656487 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:44:34.656460 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc69ecc0a_82dd_4b9a_a242_05467c3a2e66.slice/crio-0b94d6a1b7fc7a21494d9a4d230ac13f2fa306c550c4971738560bef5f5b24ca WatchSource:0}: Error finding container 0b94d6a1b7fc7a21494d9a4d230ac13f2fa306c550c4971738560bef5f5b24ca: Status 404 returned error can't find the container with id 0b94d6a1b7fc7a21494d9a4d230ac13f2fa306c550c4971738560bef5f5b24ca Jul 15 21:44:34.657802 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:34.657776 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 21:44:35.274225 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:35.274165 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051140803504" event={"ID":"c69ecc0a-82dd-4b9a-a242-05467c3a2e66","Type":"ContainerStarted","Data":"0b94d6a1b7fc7a21494d9a4d230ac13f2fa306c550c4971738560bef5f5b24ca"} Jul 15 21:44:36.281365 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:36.281327 2573 generic.go:356] "Generic (PLEG): container finished" podID="c69ecc0a-82dd-4b9a-a242-05467c3a2e66" containerID="01579c875eb1336722dc59686ae77067cf729bf9314954618e4dca8c0bbb6f41" exitCode=0 Jul 15 21:44:36.281756 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:36.281388 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051140803504" event={"ID":"c69ecc0a-82dd-4b9a-a242-05467c3a2e66","Type":"ContainerDied","Data":"01579c875eb1336722dc59686ae77067cf729bf9314954618e4dca8c0bbb6f41"} Jul 15 21:44:37.429889 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.429860 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051140803504" Jul 15 21:44:37.475285 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.475260 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7272-140051140803504_c69ecc0a-82dd-4b9a-a242-05467c3a2e66/test-curl-7272-140051140803504/0.log" Jul 15 21:44:37.501222 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.501164 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7272-140051140803504"] Jul 15 21:44:37.505001 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.504977 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7272-140051140803504"] Jul 15 21:44:37.550268 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.550149 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-479sf\" (UniqueName: \"kubernetes.io/projected/c69ecc0a-82dd-4b9a-a242-05467c3a2e66-kube-api-access-479sf\") pod \"c69ecc0a-82dd-4b9a-a242-05467c3a2e66\" (UID: \"c69ecc0a-82dd-4b9a-a242-05467c3a2e66\") " Jul 15 21:44:37.552854 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.552813 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c69ecc0a-82dd-4b9a-a242-05467c3a2e66-kube-api-access-479sf" (OuterVolumeSpecName: "kube-api-access-479sf") pod "c69ecc0a-82dd-4b9a-a242-05467c3a2e66" (UID: "c69ecc0a-82dd-4b9a-a242-05467c3a2e66"). InnerVolumeSpecName "kube-api-access-479sf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:44:37.623719 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.623685 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7272-140051140802352"] Jul 15 21:44:37.624144 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.624129 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69ecc0a-82dd-4b9a-a242-05467c3a2e66" containerName="test-curl-7272-140051140803504" Jul 15 21:44:37.624216 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.624148 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69ecc0a-82dd-4b9a-a242-05467c3a2e66" containerName="test-curl-7272-140051140803504" Jul 15 21:44:37.624267 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.624257 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c69ecc0a-82dd-4b9a-a242-05467c3a2e66" containerName="test-curl-7272-140051140803504" Jul 15 21:44:37.627402 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.627385 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051140802352" Jul 15 21:44:37.635858 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.635834 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051140802352"] Jul 15 21:44:37.651240 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.651216 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-479sf\" (UniqueName: \"kubernetes.io/projected/c69ecc0a-82dd-4b9a-a242-05467c3a2e66-kube-api-access-479sf\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:44:37.752023 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.751991 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qctpz\" (UniqueName: \"kubernetes.io/projected/aa7060ea-959d-4db4-9921-81bc9ffaea40-kube-api-access-qctpz\") pod \"test-curl-7272-140051140802352\" (UID: \"aa7060ea-959d-4db4-9921-81bc9ffaea40\") " pod="opendatahub/test-curl-7272-140051140802352" Jul 15 21:44:37.853258 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.853131 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qctpz\" (UniqueName: \"kubernetes.io/projected/aa7060ea-959d-4db4-9921-81bc9ffaea40-kube-api-access-qctpz\") pod \"test-curl-7272-140051140802352\" (UID: \"aa7060ea-959d-4db4-9921-81bc9ffaea40\") " pod="opendatahub/test-curl-7272-140051140802352" Jul 15 21:44:37.862331 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.862298 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qctpz\" (UniqueName: \"kubernetes.io/projected/aa7060ea-959d-4db4-9921-81bc9ffaea40-kube-api-access-qctpz\") pod \"test-curl-7272-140051140802352\" (UID: \"aa7060ea-959d-4db4-9921-81bc9ffaea40\") " pod="opendatahub/test-curl-7272-140051140802352" Jul 15 21:44:37.938879 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:37.938846 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051140802352" Jul 15 21:44:38.074775 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:38.074749 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051140802352"] Jul 15 21:44:38.075924 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:44:38.075900 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa7060ea_959d_4db4_9921_81bc9ffaea40.slice/crio-9d4efd72012dd221d6da3b3d79dd21cfc49283d67b23ba82b82f0aefa8662fae WatchSource:0}: Error finding container 9d4efd72012dd221d6da3b3d79dd21cfc49283d67b23ba82b82f0aefa8662fae: Status 404 returned error can't find the container with id 9d4efd72012dd221d6da3b3d79dd21cfc49283d67b23ba82b82f0aefa8662fae Jul 15 21:44:38.293723 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:38.293684 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051140802352" event={"ID":"aa7060ea-959d-4db4-9921-81bc9ffaea40","Type":"ContainerStarted","Data":"9d4efd72012dd221d6da3b3d79dd21cfc49283d67b23ba82b82f0aefa8662fae"} Jul 15 21:44:38.294831 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:38.294813 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b94d6a1b7fc7a21494d9a4d230ac13f2fa306c550c4971738560bef5f5b24ca" Jul 15 21:44:38.294950 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:38.294833 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051140803504" Jul 15 21:44:38.305682 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:44:38.305651 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7272-140051140803504\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="c69ecc0a-82dd-4b9a-a242-05467c3a2e66" pod="opendatahub/test-curl-7272-140051140803504" Jul 15 21:44:39.302239 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:39.302203 2573 generic.go:356] "Generic (PLEG): container finished" podID="aa7060ea-959d-4db4-9921-81bc9ffaea40" containerID="12e8bc8525e250b59ceec960cd7004fb67df6b05fa830a825a9a18a7c810dc4a" exitCode=0 Jul 15 21:44:39.302778 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:39.302286 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051140802352" event={"ID":"aa7060ea-959d-4db4-9921-81bc9ffaea40","Type":"ContainerDied","Data":"12e8bc8525e250b59ceec960cd7004fb67df6b05fa830a825a9a18a7c810dc4a"} Jul 15 21:44:39.304651 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:44:39.304630 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7272-140051140803504\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="c69ecc0a-82dd-4b9a-a242-05467c3a2e66" pod="opendatahub/test-curl-7272-140051140803504" Jul 15 21:44:39.449264 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:39.449226 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c69ecc0a-82dd-4b9a-a242-05467c3a2e66" path="/var/lib/kubelet/pods/c69ecc0a-82dd-4b9a-a242-05467c3a2e66/volumes" Jul 15 21:44:40.445982 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.445959 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051140802352" Jul 15 21:44:40.468672 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.468650 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7272-140051140802352_aa7060ea-959d-4db4-9921-81bc9ffaea40/test-curl-7272-140051140802352/0.log" Jul 15 21:44:40.495302 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.495274 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7272-140051140802352"] Jul 15 21:44:40.499720 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.499700 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7272-140051140802352"] Jul 15 21:44:40.581254 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.581155 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qctpz\" (UniqueName: \"kubernetes.io/projected/aa7060ea-959d-4db4-9921-81bc9ffaea40-kube-api-access-qctpz\") pod \"aa7060ea-959d-4db4-9921-81bc9ffaea40\" (UID: \"aa7060ea-959d-4db4-9921-81bc9ffaea40\") " Jul 15 21:44:40.583264 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.583240 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa7060ea-959d-4db4-9921-81bc9ffaea40-kube-api-access-qctpz" (OuterVolumeSpecName: "kube-api-access-qctpz") pod "aa7060ea-959d-4db4-9921-81bc9ffaea40" (UID: "aa7060ea-959d-4db4-9921-81bc9ffaea40"). InnerVolumeSpecName "kube-api-access-qctpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:44:40.662107 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.662075 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:40.662539 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.662526 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa7060ea-959d-4db4-9921-81bc9ffaea40" containerName="test-curl-7272-140051140802352" Jul 15 21:44:40.662586 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.662541 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa7060ea-959d-4db4-9921-81bc9ffaea40" containerName="test-curl-7272-140051140802352" Jul 15 21:44:40.662625 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.662610 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa7060ea-959d-4db4-9921-81bc9ffaea40" containerName="test-curl-7272-140051140802352" Jul 15 21:44:40.665424 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.665408 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:40.678246 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.676051 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:40.682590 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.682564 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qctpz\" (UniqueName: \"kubernetes.io/projected/aa7060ea-959d-4db4-9921-81bc9ffaea40-kube-api-access-qctpz\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:44:40.783939 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.783901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frwh9\" (UniqueName: \"kubernetes.io/projected/834209fe-5ca1-4006-8c52-6173177fc6a9-kube-api-access-frwh9\") pod \"test-curl-7272-140051141112720\" (UID: \"834209fe-5ca1-4006-8c52-6173177fc6a9\") " pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:40.885234 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.885150 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frwh9\" (UniqueName: \"kubernetes.io/projected/834209fe-5ca1-4006-8c52-6173177fc6a9-kube-api-access-frwh9\") pod \"test-curl-7272-140051141112720\" (UID: \"834209fe-5ca1-4006-8c52-6173177fc6a9\") " pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:40.894321 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.894294 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frwh9\" (UniqueName: \"kubernetes.io/projected/834209fe-5ca1-4006-8c52-6173177fc6a9-kube-api-access-frwh9\") pod \"test-curl-7272-140051141112720\" (UID: \"834209fe-5ca1-4006-8c52-6173177fc6a9\") " pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:40.981571 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:40.981540 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:41.119689 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:41.119665 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:41.121127 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:44:41.121102 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod834209fe_5ca1_4006_8c52_6173177fc6a9.slice/crio-9ebc6bc4ede13c8615b4a951d5ab8309fa141ee397b1243cbd188e3810662edf WatchSource:0}: Error finding container 9ebc6bc4ede13c8615b4a951d5ab8309fa141ee397b1243cbd188e3810662edf: Status 404 returned error can't find the container with id 9ebc6bc4ede13c8615b4a951d5ab8309fa141ee397b1243cbd188e3810662edf Jul 15 21:44:41.315320 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:41.315291 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d4efd72012dd221d6da3b3d79dd21cfc49283d67b23ba82b82f0aefa8662fae" Jul 15 21:44:41.315320 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:41.315301 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051140802352" Jul 15 21:44:41.316494 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:41.316467 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051141112720" event={"ID":"834209fe-5ca1-4006-8c52-6173177fc6a9","Type":"ContainerStarted","Data":"9ebc6bc4ede13c8615b4a951d5ab8309fa141ee397b1243cbd188e3810662edf"} Jul 15 21:44:41.326554 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:44:41.326528 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7272-140051140802352\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="aa7060ea-959d-4db4-9921-81bc9ffaea40" pod="opendatahub/test-curl-7272-140051140802352" Jul 15 21:44:41.449248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:41.449225 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa7060ea-959d-4db4-9921-81bc9ffaea40" path="/var/lib/kubelet/pods/aa7060ea-959d-4db4-9921-81bc9ffaea40/volumes" Jul 15 21:44:42.324301 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:42.324269 2573 generic.go:356] "Generic (PLEG): container finished" podID="834209fe-5ca1-4006-8c52-6173177fc6a9" containerID="778f61a8853b4767aace51044bccbe529bdea4311825be98fd91aaa48c2842ae" exitCode=0 Jul 15 21:44:42.324480 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:42.324359 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051141112720" event={"ID":"834209fe-5ca1-4006-8c52-6173177fc6a9","Type":"ContainerDied","Data":"778f61a8853b4767aace51044bccbe529bdea4311825be98fd91aaa48c2842ae"} Jul 15 21:44:43.467316 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.467289 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:43.491215 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.491188 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7272-140051141112720_834209fe-5ca1-4006-8c52-6173177fc6a9/test-curl-7272-140051141112720/0.log" Jul 15 21:44:43.509765 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.509736 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frwh9\" (UniqueName: \"kubernetes.io/projected/834209fe-5ca1-4006-8c52-6173177fc6a9-kube-api-access-frwh9\") pod \"834209fe-5ca1-4006-8c52-6173177fc6a9\" (UID: \"834209fe-5ca1-4006-8c52-6173177fc6a9\") " Jul 15 21:44:43.511920 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.511897 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834209fe-5ca1-4006-8c52-6173177fc6a9-kube-api-access-frwh9" (OuterVolumeSpecName: "kube-api-access-frwh9") pod "834209fe-5ca1-4006-8c52-6173177fc6a9" (UID: "834209fe-5ca1-4006-8c52-6173177fc6a9"). InnerVolumeSpecName "kube-api-access-frwh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:44:43.516497 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.516476 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:43.520954 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.520936 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:43.610708 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.610624 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frwh9\" (UniqueName: \"kubernetes.io/projected/834209fe-5ca1-4006-8c52-6173177fc6a9-kube-api-access-frwh9\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:44:43.637495 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.637458 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:43.638100 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.638083 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834209fe-5ca1-4006-8c52-6173177fc6a9" containerName="test-curl-7272-140051141112720" Jul 15 21:44:43.638147 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.638103 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="834209fe-5ca1-4006-8c52-6173177fc6a9" containerName="test-curl-7272-140051141112720" Jul 15 21:44:43.638244 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.638233 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="834209fe-5ca1-4006-8c52-6173177fc6a9" containerName="test-curl-7272-140051141112720" Jul 15 21:44:43.641294 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.641267 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:43.648276 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.648232 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:43.711752 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.711709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bsjp\" (UniqueName: \"kubernetes.io/projected/81f17d76-0038-4d39-8f8f-f91b4160ded9-kube-api-access-4bsjp\") pod \"test-curl-7272-140051141112720\" (UID: \"81f17d76-0038-4d39-8f8f-f91b4160ded9\") " pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:43.812443 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.812411 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bsjp\" (UniqueName: \"kubernetes.io/projected/81f17d76-0038-4d39-8f8f-f91b4160ded9-kube-api-access-4bsjp\") pod \"test-curl-7272-140051141112720\" (UID: \"81f17d76-0038-4d39-8f8f-f91b4160ded9\") " pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:43.821682 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.821648 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bsjp\" (UniqueName: \"kubernetes.io/projected/81f17d76-0038-4d39-8f8f-f91b4160ded9-kube-api-access-4bsjp\") pod \"test-curl-7272-140051141112720\" (UID: \"81f17d76-0038-4d39-8f8f-f91b4160ded9\") " pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:43.951415 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:43.951336 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:44.104724 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:44:44.104692 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81f17d76_0038_4d39_8f8f_f91b4160ded9.slice/crio-4bbf61c50762f38af03d107b7a833000eb3b22f6b7cbd96d2e430dcf0995c99b WatchSource:0}: Error finding container 4bbf61c50762f38af03d107b7a833000eb3b22f6b7cbd96d2e430dcf0995c99b: Status 404 returned error can't find the container with id 4bbf61c50762f38af03d107b7a833000eb3b22f6b7cbd96d2e430dcf0995c99b Jul 15 21:44:44.104724 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:44.104706 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:44.338292 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:44.338269 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:44.338504 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:44.338268 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ebc6bc4ede13c8615b4a951d5ab8309fa141ee397b1243cbd188e3810662edf" Jul 15 21:44:44.339540 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:44.339517 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051141112720" event={"ID":"81f17d76-0038-4d39-8f8f-f91b4160ded9","Type":"ContainerStarted","Data":"4bbf61c50762f38af03d107b7a833000eb3b22f6b7cbd96d2e430dcf0995c99b"} Jul 15 21:44:44.350921 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:44.350891 2573 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7272-140051141112720" oldPodUID="834209fe-5ca1-4006-8c52-6173177fc6a9" podUID="81f17d76-0038-4d39-8f8f-f91b4160ded9" Jul 15 21:44:45.346735 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:45.346692 2573 generic.go:356] "Generic (PLEG): container finished" podID="81f17d76-0038-4d39-8f8f-f91b4160ded9" containerID="41fd8c9bdc73d2314ad9e6fae53c535e01166e17452042b886a9aa7938aa5465" exitCode=0 Jul 15 21:44:45.347124 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:45.346780 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051141112720" event={"ID":"81f17d76-0038-4d39-8f8f-f91b4160ded9","Type":"ContainerDied","Data":"41fd8c9bdc73d2314ad9e6fae53c535e01166e17452042b886a9aa7938aa5465"} Jul 15 21:44:45.450873 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:45.450831 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="834209fe-5ca1-4006-8c52-6173177fc6a9" path="/var/lib/kubelet/pods/834209fe-5ca1-4006-8c52-6173177fc6a9/volumes" Jul 15 21:44:46.491005 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.490984 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:46.517092 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.517060 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7272-140051141112720_81f17d76-0038-4d39-8f8f-f91b4160ded9/test-curl-7272-140051141112720/0.log" Jul 15 21:44:46.538422 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.538397 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bsjp\" (UniqueName: \"kubernetes.io/projected/81f17d76-0038-4d39-8f8f-f91b4160ded9-kube-api-access-4bsjp\") pod \"81f17d76-0038-4d39-8f8f-f91b4160ded9\" (UID: \"81f17d76-0038-4d39-8f8f-f91b4160ded9\") " Jul 15 21:44:46.540855 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.540829 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f17d76-0038-4d39-8f8f-f91b4160ded9-kube-api-access-4bsjp" (OuterVolumeSpecName: "kube-api-access-4bsjp") pod "81f17d76-0038-4d39-8f8f-f91b4160ded9" (UID: "81f17d76-0038-4d39-8f8f-f91b4160ded9"). InnerVolumeSpecName "kube-api-access-4bsjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:44:46.543071 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.543019 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:46.547248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.547098 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7272-140051141112720"] Jul 15 21:44:46.639441 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.639360 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bsjp\" (UniqueName: \"kubernetes.io/projected/81f17d76-0038-4d39-8f8f-f91b4160ded9-kube-api-access-4bsjp\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:44:46.677640 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.677612 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:46.678035 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.678023 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81f17d76-0038-4d39-8f8f-f91b4160ded9" containerName="test-curl-7272-140051141112720" Jul 15 21:44:46.678087 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.678036 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="81f17d76-0038-4d39-8f8f-f91b4160ded9" containerName="test-curl-7272-140051141112720" Jul 15 21:44:46.678128 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.678110 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="81f17d76-0038-4d39-8f8f-f91b4160ded9" containerName="test-curl-7272-140051141112720" Jul 15 21:44:46.681324 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.681308 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:46.690403 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.690377 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:46.739808 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.739784 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsrqp\" (UniqueName: \"kubernetes.io/projected/127cb1d1-8169-4cff-98e6-bac5e7a496ea-kube-api-access-jsrqp\") pod \"test-curl-7272-140051141113728\" (UID: \"127cb1d1-8169-4cff-98e6-bac5e7a496ea\") " pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:46.840649 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.840610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsrqp\" (UniqueName: \"kubernetes.io/projected/127cb1d1-8169-4cff-98e6-bac5e7a496ea-kube-api-access-jsrqp\") pod \"test-curl-7272-140051141113728\" (UID: \"127cb1d1-8169-4cff-98e6-bac5e7a496ea\") " pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:46.850660 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.850633 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsrqp\" (UniqueName: \"kubernetes.io/projected/127cb1d1-8169-4cff-98e6-bac5e7a496ea-kube-api-access-jsrqp\") pod \"test-curl-7272-140051141113728\" (UID: \"127cb1d1-8169-4cff-98e6-bac5e7a496ea\") " pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:46.991666 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:46.991577 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:47.135232 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:47.135204 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:47.137737 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:44:47.137712 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod127cb1d1_8169_4cff_98e6_bac5e7a496ea.slice/crio-37aef2deeb50ba34b4675cbc9c7e77a2bba767cb9fd6e195cb8c62f30bde287d WatchSource:0}: Error finding container 37aef2deeb50ba34b4675cbc9c7e77a2bba767cb9fd6e195cb8c62f30bde287d: Status 404 returned error can't find the container with id 37aef2deeb50ba34b4675cbc9c7e77a2bba767cb9fd6e195cb8c62f30bde287d Jul 15 21:44:47.360704 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:47.360673 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:47.360704 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:47.360706 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bbf61c50762f38af03d107b7a833000eb3b22f6b7cbd96d2e430dcf0995c99b" Jul 15 21:44:47.361970 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:47.361943 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051141113728" event={"ID":"127cb1d1-8169-4cff-98e6-bac5e7a496ea","Type":"ContainerStarted","Data":"37aef2deeb50ba34b4675cbc9c7e77a2bba767cb9fd6e195cb8c62f30bde287d"} Jul 15 21:44:47.371839 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:44:47.371812 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7272-140051141112720\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="81f17d76-0038-4d39-8f8f-f91b4160ded9" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:47.446684 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:44:47.446649 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7272-140051141112720\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="81f17d76-0038-4d39-8f8f-f91b4160ded9" pod="opendatahub/test-curl-7272-140051141112720" Jul 15 21:44:47.448121 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:47.448096 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81f17d76-0038-4d39-8f8f-f91b4160ded9" path="/var/lib/kubelet/pods/81f17d76-0038-4d39-8f8f-f91b4160ded9/volumes" Jul 15 21:44:48.368925 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:48.368894 2573 generic.go:356] "Generic (PLEG): container finished" podID="127cb1d1-8169-4cff-98e6-bac5e7a496ea" containerID="db238e1a400a72da2d912d458b66d939435c06b78e943c921b33262cc7e774d8" exitCode=0 Jul 15 21:44:48.369361 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:48.368949 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051141113728" event={"ID":"127cb1d1-8169-4cff-98e6-bac5e7a496ea","Type":"ContainerDied","Data":"db238e1a400a72da2d912d458b66d939435c06b78e943c921b33262cc7e774d8"} Jul 15 21:44:49.515067 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.515044 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:49.537194 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.537129 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7272-140051141113728_127cb1d1-8169-4cff-98e6-bac5e7a496ea/test-curl-7272-140051141113728/0.log" Jul 15 21:44:49.561504 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.561463 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:49.567775 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.567751 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:49.570269 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.570239 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsrqp\" (UniqueName: \"kubernetes.io/projected/127cb1d1-8169-4cff-98e6-bac5e7a496ea-kube-api-access-jsrqp\") pod \"127cb1d1-8169-4cff-98e6-bac5e7a496ea\" (UID: \"127cb1d1-8169-4cff-98e6-bac5e7a496ea\") " Jul 15 21:44:49.572290 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.572270 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/127cb1d1-8169-4cff-98e6-bac5e7a496ea-kube-api-access-jsrqp" (OuterVolumeSpecName: "kube-api-access-jsrqp") pod "127cb1d1-8169-4cff-98e6-bac5e7a496ea" (UID: "127cb1d1-8169-4cff-98e6-bac5e7a496ea"). InnerVolumeSpecName "kube-api-access-jsrqp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:44:49.671483 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.671406 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsrqp\" (UniqueName: \"kubernetes.io/projected/127cb1d1-8169-4cff-98e6-bac5e7a496ea-kube-api-access-jsrqp\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:44:49.690156 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.690129 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:49.690635 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.690619 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127cb1d1-8169-4cff-98e6-bac5e7a496ea" containerName="test-curl-7272-140051141113728" Jul 15 21:44:49.690681 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.690637 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="127cb1d1-8169-4cff-98e6-bac5e7a496ea" containerName="test-curl-7272-140051141113728" Jul 15 21:44:49.690713 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.690706 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="127cb1d1-8169-4cff-98e6-bac5e7a496ea" containerName="test-curl-7272-140051141113728" Jul 15 21:44:49.693364 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.693349 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:49.701665 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.701641 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:49.772697 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.772664 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fd4r\" (UniqueName: \"kubernetes.io/projected/5a78d1d4-bec7-4c65-9251-de14172d0708-kube-api-access-6fd4r\") pod \"test-curl-7272-140051141113728\" (UID: \"5a78d1d4-bec7-4c65-9251-de14172d0708\") " pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:49.874187 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.874140 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fd4r\" (UniqueName: \"kubernetes.io/projected/5a78d1d4-bec7-4c65-9251-de14172d0708-kube-api-access-6fd4r\") pod \"test-curl-7272-140051141113728\" (UID: \"5a78d1d4-bec7-4c65-9251-de14172d0708\") " pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:49.884006 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:49.883984 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fd4r\" (UniqueName: \"kubernetes.io/projected/5a78d1d4-bec7-4c65-9251-de14172d0708-kube-api-access-6fd4r\") pod \"test-curl-7272-140051141113728\" (UID: \"5a78d1d4-bec7-4c65-9251-de14172d0708\") " pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:50.002934 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:50.002908 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:50.138788 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:50.138762 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:50.140889 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:44:50.140862 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a78d1d4_bec7_4c65_9251_de14172d0708.slice/crio-a211f8f6ec96193882cabe22f294b6721fb09daedb0d4816150b294958940c2b WatchSource:0}: Error finding container a211f8f6ec96193882cabe22f294b6721fb09daedb0d4816150b294958940c2b: Status 404 returned error can't find the container with id a211f8f6ec96193882cabe22f294b6721fb09daedb0d4816150b294958940c2b Jul 15 21:44:50.382143 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:50.382044 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051141113728" event={"ID":"5a78d1d4-bec7-4c65-9251-de14172d0708","Type":"ContainerStarted","Data":"a211f8f6ec96193882cabe22f294b6721fb09daedb0d4816150b294958940c2b"} Jul 15 21:44:50.383224 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:50.383204 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:50.383357 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:50.383236 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37aef2deeb50ba34b4675cbc9c7e77a2bba767cb9fd6e195cb8c62f30bde287d" Jul 15 21:44:50.395545 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:50.395520 2573 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7272-140051141113728" oldPodUID="127cb1d1-8169-4cff-98e6-bac5e7a496ea" podUID="5a78d1d4-bec7-4c65-9251-de14172d0708" Jul 15 21:44:51.390968 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:51.390934 2573 generic.go:356] "Generic (PLEG): container finished" podID="5a78d1d4-bec7-4c65-9251-de14172d0708" containerID="73efb196c0f8a647732ff1a2ef97480edfd6d6f99d6ffcae30e2ab8e5a993cdc" exitCode=0 Jul 15 21:44:51.391360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:51.391018 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7272-140051141113728" event={"ID":"5a78d1d4-bec7-4c65-9251-de14172d0708","Type":"ContainerDied","Data":"73efb196c0f8a647732ff1a2ef97480edfd6d6f99d6ffcae30e2ab8e5a993cdc"} Jul 15 21:44:51.447940 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:51.447901 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="127cb1d1-8169-4cff-98e6-bac5e7a496ea" path="/var/lib/kubelet/pods/127cb1d1-8169-4cff-98e6-bac5e7a496ea/volumes" Jul 15 21:44:52.550681 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.550651 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:52.576378 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.576352 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7272-140051141113728_5a78d1d4-bec7-4c65-9251-de14172d0708/test-curl-7272-140051141113728/0.log" Jul 15 21:44:52.600865 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.600831 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fd4r\" (UniqueName: \"kubernetes.io/projected/5a78d1d4-bec7-4c65-9251-de14172d0708-kube-api-access-6fd4r\") pod \"5a78d1d4-bec7-4c65-9251-de14172d0708\" (UID: \"5a78d1d4-bec7-4c65-9251-de14172d0708\") " Jul 15 21:44:52.602588 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.602564 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:52.603017 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.602995 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a78d1d4-bec7-4c65-9251-de14172d0708-kube-api-access-6fd4r" (OuterVolumeSpecName: "kube-api-access-6fd4r") pod "5a78d1d4-bec7-4c65-9251-de14172d0708" (UID: "5a78d1d4-bec7-4c65-9251-de14172d0708"). InnerVolumeSpecName "kube-api-access-6fd4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:44:52.609291 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.609249 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7272-140051141113728"] Jul 15 21:44:52.702624 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.702537 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fd4r\" (UniqueName: \"kubernetes.io/projected/5a78d1d4-bec7-4c65-9251-de14172d0708-kube-api-access-6fd4r\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:44:52.797882 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.797847 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj"] Jul 15 21:44:52.798294 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.798280 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a78d1d4-bec7-4c65-9251-de14172d0708" containerName="test-curl-7272-140051141113728" Jul 15 21:44:52.798360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.798296 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a78d1d4-bec7-4c65-9251-de14172d0708" containerName="test-curl-7272-140051141113728" Jul 15 21:44:52.798402 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.798393 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a78d1d4-bec7-4c65-9251-de14172d0708" containerName="test-curl-7272-140051141113728" Jul 15 21:44:52.801546 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.801530 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" Jul 15 21:44:52.804472 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.804450 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kpm6w\"" Jul 15 21:44:52.810046 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.810018 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj"] Jul 15 21:44:52.903715 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:52.903672 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce746de-10ed-488f-8b38-7fd1b7a50f82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj\" (UID: \"6ce746de-10ed-488f-8b38-7fd1b7a50f82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" Jul 15 21:44:53.004390 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:53.004354 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce746de-10ed-488f-8b38-7fd1b7a50f82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj\" (UID: \"6ce746de-10ed-488f-8b38-7fd1b7a50f82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" Jul 15 21:44:53.004995 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:53.004976 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce746de-10ed-488f-8b38-7fd1b7a50f82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj\" (UID: \"6ce746de-10ed-488f-8b38-7fd1b7a50f82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" Jul 15 21:44:53.113937 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:53.113895 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" Jul 15 21:44:53.259398 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:53.259369 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj"] Jul 15 21:44:53.260238 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:44:53.260208 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce746de_10ed_488f_8b38_7fd1b7a50f82.slice/crio-731c67624b69dfb7d7373edcef15453f40d378d106087173a6d73b611a52fbb2 WatchSource:0}: Error finding container 731c67624b69dfb7d7373edcef15453f40d378d106087173a6d73b611a52fbb2: Status 404 returned error can't find the container with id 731c67624b69dfb7d7373edcef15453f40d378d106087173a6d73b611a52fbb2 Jul 15 21:44:53.406972 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:53.406942 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a211f8f6ec96193882cabe22f294b6721fb09daedb0d4816150b294958940c2b" Jul 15 21:44:53.406972 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:53.406971 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:53.408423 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:53.408385 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" event={"ID":"6ce746de-10ed-488f-8b38-7fd1b7a50f82","Type":"ContainerStarted","Data":"731c67624b69dfb7d7373edcef15453f40d378d106087173a6d73b611a52fbb2"} Jul 15 21:44:53.419562 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:44:53.419534 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7272-140051141113728\" is forbidden: User \"system:node:ip-10-0-135-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-80.ec2.internal' and this object" podUID="5a78d1d4-bec7-4c65-9251-de14172d0708" pod="opendatahub/test-curl-7272-140051141113728" Jul 15 21:44:53.450524 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:53.450495 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a78d1d4-bec7-4c65-9251-de14172d0708" path="/var/lib/kubelet/pods/5a78d1d4-bec7-4c65-9251-de14172d0708/volumes" Jul 15 21:44:54.416593 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:54.416555 2573 generic.go:356] "Generic (PLEG): container finished" podID="6ce746de-10ed-488f-8b38-7fd1b7a50f82" containerID="21bae871c53c05d6dc52d41736e285e4dd9d1f46493a6ce316e977e332d96c79" exitCode=0 Jul 15 21:44:54.416941 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:54.416634 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" event={"ID":"6ce746de-10ed-488f-8b38-7fd1b7a50f82","Type":"ContainerDied","Data":"21bae871c53c05d6dc52d41736e285e4dd9d1f46493a6ce316e977e332d96c79"} Jul 15 21:44:55.577307 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:55.577283 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" Jul 15 21:44:55.647117 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:55.647085 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce746de-10ed-488f-8b38-7fd1b7a50f82-maas-api-service-ca\") pod \"6ce746de-10ed-488f-8b38-7fd1b7a50f82\" (UID: \"6ce746de-10ed-488f-8b38-7fd1b7a50f82\") " Jul 15 21:44:55.647472 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:55.647450 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ce746de-10ed-488f-8b38-7fd1b7a50f82-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ce746de-10ed-488f-8b38-7fd1b7a50f82" (UID: "6ce746de-10ed-488f-8b38-7fd1b7a50f82"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:44:55.748676 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:55.748601 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ce746de-10ed-488f-8b38-7fd1b7a50f82-maas-api-service-ca\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:44:56.431413 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:56.431381 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" event={"ID":"6ce746de-10ed-488f-8b38-7fd1b7a50f82","Type":"ContainerDied","Data":"731c67624b69dfb7d7373edcef15453f40d378d106087173a6d73b611a52fbb2"} Jul 15 21:44:56.431582 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:56.431423 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj" Jul 15 21:44:56.431582 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:44:56.431430 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="731c67624b69dfb7d7373edcef15453f40d378d106087173a6d73b611a52fbb2" Jul 15 21:45:00.148777 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.148735 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m"] Jul 15 21:45:00.149384 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.149364 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ce746de-10ed-488f-8b38-7fd1b7a50f82" containerName="revoke-keys" Jul 15 21:45:00.149490 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.149386 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ce746de-10ed-488f-8b38-7fd1b7a50f82" containerName="revoke-keys" Jul 15 21:45:00.149548 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.149494 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ce746de-10ed-488f-8b38-7fd1b7a50f82" containerName="revoke-keys" Jul 15 21:45:00.152658 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.152637 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c"] Jul 15 21:45:00.152782 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.152751 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m" Jul 15 21:45:00.155988 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.155965 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kpm6w\"" Jul 15 21:45:00.156203 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.156021 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9"] Jul 15 21:45:00.156302 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.156089 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c" Jul 15 21:45:00.160382 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.160361 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m"] Jul 15 21:45:00.160495 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.160468 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9" Jul 15 21:45:00.164469 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.164212 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c"] Jul 15 21:45:00.167318 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.167297 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9"] Jul 15 21:45:00.169625 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.169608 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m" Jul 15 21:45:00.178066 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.178044 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c" Jul 15 21:45:00.220568 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.220534 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9" Jul 15 21:45:00.346454 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.345411 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m"] Jul 15 21:45:00.349457 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:45:00.349428 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f2f2c0c_7632_4dc5_8e2e_d9c6f5be4689.slice/crio-51d03b1619cc611b76cd65664dc3e49409083a52ef75769bacf048cce77da0ef WatchSource:0}: Error finding container 51d03b1619cc611b76cd65664dc3e49409083a52ef75769bacf048cce77da0ef: Status 404 returned error can't find the container with id 51d03b1619cc611b76cd65664dc3e49409083a52ef75769bacf048cce77da0ef Jul 15 21:45:00.372925 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.372900 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c"] Jul 15 21:45:00.375309 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:45:00.375285 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0dadfb2_2e7d_4887_b76e_f7ccb2caf93b.slice/crio-7ee2f78e5036eefc6e80f48c8cef514582a17912114e0aea27ab77f6d945f59a WatchSource:0}: Error finding container 7ee2f78e5036eefc6e80f48c8cef514582a17912114e0aea27ab77f6d945f59a: Status 404 returned error can't find the container with id 7ee2f78e5036eefc6e80f48c8cef514582a17912114e0aea27ab77f6d945f59a Jul 15 21:45:00.403971 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.403938 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9"] Jul 15 21:45:00.404999 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:45:00.404978 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d97fcf3_7bc5_430c_9f96_594ddaefbff5.slice/crio-776d42ddf034a10fa4df34220890b2010dc84754cff4b751a5d4dd500fc12ece WatchSource:0}: Error finding container 776d42ddf034a10fa4df34220890b2010dc84754cff4b751a5d4dd500fc12ece: Status 404 returned error can't find the container with id 776d42ddf034a10fa4df34220890b2010dc84754cff4b751a5d4dd500fc12ece Jul 15 21:45:00.456830 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.456794 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m" event={"ID":"8f2f2c0c-7632-4dc5-8e2e-d9c6f5be4689","Type":"ContainerStarted","Data":"51d03b1619cc611b76cd65664dc3e49409083a52ef75769bacf048cce77da0ef"} Jul 15 21:45:00.457936 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.457906 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c" event={"ID":"d0dadfb2-2e7d-4887-b76e-f7ccb2caf93b","Type":"ContainerStarted","Data":"7ee2f78e5036eefc6e80f48c8cef514582a17912114e0aea27ab77f6d945f59a"} Jul 15 21:45:00.459031 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:00.459011 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9" event={"ID":"7d97fcf3-7bc5-430c-9f96-594ddaefbff5","Type":"ContainerStarted","Data":"776d42ddf034a10fa4df34220890b2010dc84754cff4b751a5d4dd500fc12ece"} Jul 15 21:45:01.466835 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:01.466801 2573 generic.go:356] "Generic (PLEG): container finished" podID="8f2f2c0c-7632-4dc5-8e2e-d9c6f5be4689" containerID="12bc90b7222179a8ee97a3ec01f2e131b51de2b39e2833b01172438e43a3f218" exitCode=0 Jul 15 21:45:01.467207 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:01.466874 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m" event={"ID":"8f2f2c0c-7632-4dc5-8e2e-d9c6f5be4689","Type":"ContainerDied","Data":"12bc90b7222179a8ee97a3ec01f2e131b51de2b39e2833b01172438e43a3f218"} Jul 15 21:45:01.468384 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:01.468364 2573 generic.go:356] "Generic (PLEG): container finished" podID="d0dadfb2-2e7d-4887-b76e-f7ccb2caf93b" containerID="58415c9399ecf0f434dd85ce28a212baefb1b64bef4aa21fbfc64ac24d88b7a7" exitCode=0 Jul 15 21:45:01.468456 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:01.468438 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c" event={"ID":"d0dadfb2-2e7d-4887-b76e-f7ccb2caf93b","Type":"ContainerDied","Data":"58415c9399ecf0f434dd85ce28a212baefb1b64bef4aa21fbfc64ac24d88b7a7"} Jul 15 21:45:01.469818 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:01.469797 2573 generic.go:356] "Generic (PLEG): container finished" podID="7d97fcf3-7bc5-430c-9f96-594ddaefbff5" containerID="bdaf4231f444c14aa2958a125a74435c3c374de21ad13a51ea10091a95547fd4" exitCode=0 Jul 15 21:45:01.469914 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:01.469823 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9" event={"ID":"7d97fcf3-7bc5-430c-9f96-594ddaefbff5","Type":"ContainerDied","Data":"bdaf4231f444c14aa2958a125a74435c3c374de21ad13a51ea10091a95547fd4"} Jul 15 21:45:02.613584 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:02.613558 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c" Jul 15 21:45:02.660428 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:02.660405 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9" Jul 15 21:45:02.682360 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:02.682335 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m" Jul 15 21:45:03.367558 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.367520 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m"] Jul 15 21:45:03.372923 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.372897 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m"] Jul 15 21:45:03.448531 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.448499 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f2f2c0c-7632-4dc5-8e2e-d9c6f5be4689" path="/var/lib/kubelet/pods/8f2f2c0c-7632-4dc5-8e2e-d9c6f5be4689/volumes" Jul 15 21:45:03.482837 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.482811 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-39670c-29735865-7c29m" Jul 15 21:45:03.482978 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.482825 2573 scope.go:117] "RemoveContainer" containerID="12bc90b7222179a8ee97a3ec01f2e131b51de2b39e2833b01172438e43a3f218" Jul 15 21:45:03.484384 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.484325 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c" Jul 15 21:45:03.484384 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.484340 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c" event={"ID":"d0dadfb2-2e7d-4887-b76e-f7ccb2caf93b","Type":"ContainerDied","Data":"7ee2f78e5036eefc6e80f48c8cef514582a17912114e0aea27ab77f6d945f59a"} Jul 15 21:45:03.484629 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.484397 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ee2f78e5036eefc6e80f48c8cef514582a17912114e0aea27ab77f6d945f59a" Jul 15 21:45:03.486057 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.486039 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9" Jul 15 21:45:03.486202 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.486082 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735865-5d7k9" event={"ID":"7d97fcf3-7bc5-430c-9f96-594ddaefbff5","Type":"ContainerDied","Data":"776d42ddf034a10fa4df34220890b2010dc84754cff4b751a5d4dd500fc12ece"} Jul 15 21:45:03.486202 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:03.486137 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="776d42ddf034a10fa4df34220890b2010dc84754cff4b751a5d4dd500fc12ece" Jul 15 21:45:15.456235 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456196 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9"] Jul 15 21:45:15.456711 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456646 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f2f2c0c-7632-4dc5-8e2e-d9c6f5be4689" containerName="cleanup" Jul 15 21:45:15.456711 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456659 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f2f2c0c-7632-4dc5-8e2e-d9c6f5be4689" containerName="cleanup" Jul 15 21:45:15.456711 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456668 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d97fcf3-7bc5-430c-9f96-594ddaefbff5" containerName="cleanup" Jul 15 21:45:15.456711 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456677 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d97fcf3-7bc5-430c-9f96-594ddaefbff5" containerName="cleanup" Jul 15 21:45:15.456711 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456694 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0dadfb2-2e7d-4887-b76e-f7ccb2caf93b" containerName="cleanup" Jul 15 21:45:15.456711 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456699 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0dadfb2-2e7d-4887-b76e-f7ccb2caf93b" containerName="cleanup" Jul 15 21:45:15.456909 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456756 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f2f2c0c-7632-4dc5-8e2e-d9c6f5be4689" containerName="cleanup" Jul 15 21:45:15.456909 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456768 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0dadfb2-2e7d-4887-b76e-f7ccb2caf93b" containerName="cleanup" Jul 15 21:45:15.456909 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.456777 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d97fcf3-7bc5-430c-9f96-594ddaefbff5" containerName="cleanup" Jul 15 21:45:15.461301 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.461285 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" Jul 15 21:45:15.465066 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.465046 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kpm6w\"" Jul 15 21:45:15.472231 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.472206 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9"] Jul 15 21:45:15.539099 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.539062 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a640429e-410e-4024-bc29-aef6cc9ad182-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9\" (UID: \"a640429e-410e-4024-bc29-aef6cc9ad182\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" Jul 15 21:45:15.640762 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.640718 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a640429e-410e-4024-bc29-aef6cc9ad182-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9\" (UID: \"a640429e-410e-4024-bc29-aef6cc9ad182\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" Jul 15 21:45:15.641364 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.641346 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a640429e-410e-4024-bc29-aef6cc9ad182-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9\" (UID: \"a640429e-410e-4024-bc29-aef6cc9ad182\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" Jul 15 21:45:15.773368 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.773333 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" Jul 15 21:45:15.917599 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:15.917573 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9"] Jul 15 21:45:15.920060 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:45:15.920030 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda640429e_410e_4024_bc29_aef6cc9ad182.slice/crio-addde30ba2b098b235dd357fe14867ea9a663e0f81838c226cea0753ffb517fa WatchSource:0}: Error finding container addde30ba2b098b235dd357fe14867ea9a663e0f81838c226cea0753ffb517fa: Status 404 returned error can't find the container with id addde30ba2b098b235dd357fe14867ea9a663e0f81838c226cea0753ffb517fa Jul 15 21:45:16.568924 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:16.568875 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" event={"ID":"a640429e-410e-4024-bc29-aef6cc9ad182","Type":"ContainerStarted","Data":"fe0ae4f13d5f38074dcca01b3f606ba53bbad15ac80c774f9d365a76b58d91cc"} Jul 15 21:45:16.569442 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:16.568948 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" event={"ID":"a640429e-410e-4024-bc29-aef6cc9ad182","Type":"ContainerStarted","Data":"addde30ba2b098b235dd357fe14867ea9a663e0f81838c226cea0753ffb517fa"} Jul 15 21:45:16.590776 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:16.590720 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" podStartSLOduration=1.5907054 podStartE2EDuration="1.5907054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:45:16.589001935 +0000 UTC m=+2769.679235117" watchObservedRunningTime="2026-07-15 21:45:16.5907054 +0000 UTC m=+2769.680938604" Jul 15 21:45:17.577475 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:17.577441 2573 generic.go:356] "Generic (PLEG): container finished" podID="a640429e-410e-4024-bc29-aef6cc9ad182" containerID="fe0ae4f13d5f38074dcca01b3f606ba53bbad15ac80c774f9d365a76b58d91cc" exitCode=0 Jul 15 21:45:17.577886 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:17.577534 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" event={"ID":"a640429e-410e-4024-bc29-aef6cc9ad182","Type":"ContainerDied","Data":"fe0ae4f13d5f38074dcca01b3f606ba53bbad15ac80c774f9d365a76b58d91cc"} Jul 15 21:45:18.717436 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:18.717407 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" Jul 15 21:45:18.769352 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:18.769319 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a640429e-410e-4024-bc29-aef6cc9ad182-maas-api-service-ca\") pod \"a640429e-410e-4024-bc29-aef6cc9ad182\" (UID: \"a640429e-410e-4024-bc29-aef6cc9ad182\") " Jul 15 21:45:18.769709 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:18.769679 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a640429e-410e-4024-bc29-aef6cc9ad182-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a640429e-410e-4024-bc29-aef6cc9ad182" (UID: "a640429e-410e-4024-bc29-aef6cc9ad182"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:45:18.870104 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:18.870003 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a640429e-410e-4024-bc29-aef6cc9ad182-maas-api-service-ca\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:45:19.592331 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:19.592302 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" Jul 15 21:45:19.592331 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:19.592329 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9" event={"ID":"a640429e-410e-4024-bc29-aef6cc9ad182","Type":"ContainerDied","Data":"addde30ba2b098b235dd357fe14867ea9a663e0f81838c226cea0753ffb517fa"} Jul 15 21:45:19.592558 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:19.592372 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="addde30ba2b098b235dd357fe14867ea9a663e0f81838c226cea0753ffb517fa" Jul 15 21:45:25.995781 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:25.995747 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5"] Jul 15 21:45:25.996576 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:25.996511 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" podUID="100a016b-8477-4085-a424-980b2b3a5e9b" containerName="maas-api" containerID="cri-o://84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85" gracePeriod=30 Jul 15 21:45:26.027726 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.027678 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c"] Jul 15 21:45:26.030415 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.030390 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-feaa76-29735865-wmf7c"] Jul 15 21:45:26.260249 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.260225 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:45:26.342944 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.342914 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrncr\" (UniqueName: \"kubernetes.io/projected/100a016b-8477-4085-a424-980b2b3a5e9b-kube-api-access-vrncr\") pod \"100a016b-8477-4085-a424-980b2b3a5e9b\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " Jul 15 21:45:26.342944 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.342949 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/100a016b-8477-4085-a424-980b2b3a5e9b-maas-api-tls\") pod \"100a016b-8477-4085-a424-980b2b3a5e9b\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " Jul 15 21:45:26.343220 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.342985 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/100a016b-8477-4085-a424-980b2b3a5e9b-openshift-service-ca\") pod \"100a016b-8477-4085-a424-980b2b3a5e9b\" (UID: \"100a016b-8477-4085-a424-980b2b3a5e9b\") " Jul 15 21:45:26.343418 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.343388 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/100a016b-8477-4085-a424-980b2b3a5e9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "100a016b-8477-4085-a424-980b2b3a5e9b" (UID: "100a016b-8477-4085-a424-980b2b3a5e9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 21:45:26.345140 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.345122 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/100a016b-8477-4085-a424-980b2b3a5e9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "100a016b-8477-4085-a424-980b2b3a5e9b" (UID: "100a016b-8477-4085-a424-980b2b3a5e9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 21:45:26.345311 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.345290 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/100a016b-8477-4085-a424-980b2b3a5e9b-kube-api-access-vrncr" (OuterVolumeSpecName: "kube-api-access-vrncr") pod "100a016b-8477-4085-a424-980b2b3a5e9b" (UID: "100a016b-8477-4085-a424-980b2b3a5e9b"). InnerVolumeSpecName "kube-api-access-vrncr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 21:45:26.444178 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.444143 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrncr\" (UniqueName: \"kubernetes.io/projected/100a016b-8477-4085-a424-980b2b3a5e9b-kube-api-access-vrncr\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:45:26.444343 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.444188 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/100a016b-8477-4085-a424-980b2b3a5e9b-maas-api-tls\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:45:26.444343 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.444198 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/100a016b-8477-4085-a424-980b2b3a5e9b-openshift-service-ca\") on node \"ip-10-0-135-80.ec2.internal\" DevicePath \"\"" Jul 15 21:45:26.636099 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.636008 2573 generic.go:356] "Generic (PLEG): container finished" podID="100a016b-8477-4085-a424-980b2b3a5e9b" containerID="84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85" exitCode=0 Jul 15 21:45:26.636099 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.636046 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" Jul 15 21:45:26.636099 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.636055 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" event={"ID":"100a016b-8477-4085-a424-980b2b3a5e9b","Type":"ContainerDied","Data":"84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85"} Jul 15 21:45:26.636099 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.636097 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5" event={"ID":"100a016b-8477-4085-a424-980b2b3a5e9b","Type":"ContainerDied","Data":"5fd096908a592740ae7c98be6d56d42fd1e0eb712984508bb189996274e8d7d9"} Jul 15 21:45:26.636539 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.636117 2573 scope.go:117] "RemoveContainer" containerID="84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85" Jul 15 21:45:26.646248 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.646227 2573 scope.go:117] "RemoveContainer" containerID="84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85" Jul 15 21:45:26.646540 ip-10-0-135-80 kubenswrapper[2573]: E0715 21:45:26.646519 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85\": container with ID starting with 84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85 not found: ID does not exist" containerID="84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85" Jul 15 21:45:26.646639 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.646548 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85"} err="failed to get container status \"84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85\": rpc error: code = NotFound desc = could not find container \"84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85\": container with ID starting with 84a99d756e4f6fdf8399f9c75e9c96c0a751514c732082209e3cbadd00833d85 not found: ID does not exist" Jul 15 21:45:26.659452 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.659425 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5"] Jul 15 21:45:26.665099 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:26.665072 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-feaa76-7fcbd498f4-6t2r5"] Jul 15 21:45:27.447726 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:27.447696 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="100a016b-8477-4085-a424-980b2b3a5e9b" path="/var/lib/kubelet/pods/100a016b-8477-4085-a424-980b2b3a5e9b/volumes" Jul 15 21:45:27.448096 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:27.447985 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0dadfb2-2e7d-4887-b76e-f7ccb2caf93b" path="/var/lib/kubelet/pods/d0dadfb2-2e7d-4887-b76e-f7ccb2caf93b/volumes" Jul 15 21:45:44.170981 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:44.170903 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-59d5c68977-fhdzm_c3ca1567-0a3b-49e8-b166-c368e2d87553/manager/0.log" Jul 15 21:45:44.282254 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:44.282220 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ccf85_85601b0c-aa09-4e72-a515-784c4d9661ba/server/0.log" Jul 15 21:45:44.769686 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:44.769656 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74776f8bd9-xm2jb_7c6d6487-2ef6-44c1-b438-cc3c92ad3797/copy-manifests/0.log" Jul 15 21:45:44.776327 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:44.776302 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74776f8bd9-xm2jb_7c6d6487-2ef6-44c1-b438-cc3c92ad3797/manager/0.log" Jul 15 21:45:45.131620 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:45.131546 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7f9664fdf-lrldp_5d429723-8df7-4cf2-8794-1b75262b7121/maas-api/0.log" Jul 15 21:45:45.256694 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:45.256668 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735835-blmh9_c23ffc78-e3d8-4ace-9603-eaa5290752c0/cleanup/0.log" Jul 15 21:45:45.478068 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:45.477982 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735865-5d7k9_7d97fcf3-7bc5-430c-9f96-594ddaefbff5/cleanup/0.log" Jul 15 21:45:45.820019 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:45.819994 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-a05ec7-d9k88_7ea91beb-6cb5-44bc-9f74-9e17c8b1ed22/revoke-keys/0.log" Jul 15 21:45:45.931373 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:45.931342 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-351fa3-6zrqk_32bafe0d-99ba-4ce8-94fd-0846dbcdd18b/revoke-keys/0.log" Jul 15 21:45:46.730691 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:46.730665 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-39670c-pnhdj_6ce746de-10ed-488f-8b38-7fd1b7a50f82/revoke-keys/0.log" Jul 15 21:45:46.845778 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:46.845752 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-feaa76-zrkk9_a640429e-410e-4024-bc29-aef6cc9ad182/revoke-keys/0.log" Jul 15 21:45:46.956238 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:46.956208 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-8l4wz_a74efc64-2f0d-482e-abe9-adec7fa5d89f/postgres/0.log" Jul 15 21:45:47.725078 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:47.725050 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz_1fe444cd-bc64-49e6-a643-f175f8840a7c/util/0.log" Jul 15 21:45:47.732271 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:47.732249 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz_1fe444cd-bc64-49e6-a643-f175f8840a7c/pull/0.log" Jul 15 21:45:47.739106 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:47.739087 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz_1fe444cd-bc64-49e6-a643-f175f8840a7c/extract/0.log" Jul 15 21:45:47.848267 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:47.848243 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q_9450047d-2a40-4ea7-a887-9dbe4c48269d/pull/0.log" Jul 15 21:45:47.854553 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:47.854536 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q_9450047d-2a40-4ea7-a887-9dbe4c48269d/extract/0.log" Jul 15 21:45:47.860148 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:47.860126 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q_9450047d-2a40-4ea7-a887-9dbe4c48269d/util/0.log" Jul 15 21:45:47.972308 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:47.972283 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9_1348b943-daec-459c-9b72-cd854faa5e8b/util/0.log" Jul 15 21:45:47.978705 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:47.978643 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9_1348b943-daec-459c-9b72-cd854faa5e8b/pull/0.log" Jul 15 21:45:47.984575 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:47.984553 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9_1348b943-daec-459c-9b72-cd854faa5e8b/extract/0.log" Jul 15 21:45:48.095293 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:48.095262 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq_97209923-879b-4976-a942-b09d8672b61f/util/0.log" Jul 15 21:45:48.103425 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:48.103401 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq_97209923-879b-4976-a942-b09d8672b61f/pull/0.log" Jul 15 21:45:48.109611 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:48.109588 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq_97209923-879b-4976-a942-b09d8672b61f/extract/0.log" Jul 15 21:45:48.465223 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:48.465191 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-89h59_01f63185-f7d1-4781-b61a-58705dc8439b/manager/0.log" Jul 15 21:45:48.699339 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:48.699305 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-56hfv_c86f5dcf-93c5-428a-90f7-3f7f710b3d5b/registry-server/0.log" Jul 15 21:45:48.827678 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:48.827640 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jwdsh_64f52d57-004a-4fee-a8f1-36e939ccab01/manager/0.log" Jul 15 21:45:49.398034 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:49.398005 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46vn9s_36b9ccbb-9fa2-4561-b234-49b301aeee60/istio-proxy/0.log" Jul 15 21:45:49.870546 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:49.870500 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qf9dp_cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7/istio-proxy/1.log" Jul 15 21:45:50.103871 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:50.103842 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-l6hpz_8cf3a567-8ba9-41a3-8b71-9296626ccf35/payload-processing/0.log" Jul 15 21:45:50.218616 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:50.218541 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585bb7b99d-xcp26_14cd8b87-f21c-41c3-8cb8-7893d3f5fa28/router/0.log" Jul 15 21:45:50.778072 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:50.778047 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-74gzw_40f9af7b-827b-40ab-97f4-0ca2f59f8747/main/0.log" Jul 15 21:45:57.322751 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.322715 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dprpk/must-gather-bnv4d"] Jul 15 21:45:57.323203 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.323189 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="100a016b-8477-4085-a424-980b2b3a5e9b" containerName="maas-api" Jul 15 21:45:57.323203 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.323205 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="100a016b-8477-4085-a424-980b2b3a5e9b" containerName="maas-api" Jul 15 21:45:57.323287 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.323214 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a640429e-410e-4024-bc29-aef6cc9ad182" containerName="revoke-keys" Jul 15 21:45:57.323287 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.323220 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a640429e-410e-4024-bc29-aef6cc9ad182" containerName="revoke-keys" Jul 15 21:45:57.323358 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.323306 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a640429e-410e-4024-bc29-aef6cc9ad182" containerName="revoke-keys" Jul 15 21:45:57.323358 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.323315 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="100a016b-8477-4085-a424-980b2b3a5e9b" containerName="maas-api" Jul 15 21:45:57.326487 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.326469 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dprpk/must-gather-bnv4d" Jul 15 21:45:57.329707 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.329681 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dprpk\"/\"openshift-service-ca.crt\"" Jul 15 21:45:57.330809 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.330790 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dprpk\"/\"default-dockercfg-tsvf8\"" Jul 15 21:45:57.330910 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.330818 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dprpk\"/\"kube-root-ca.crt\"" Jul 15 21:45:57.336291 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.336269 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dprpk/must-gather-bnv4d"] Jul 15 21:45:57.436328 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.436297 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/74e4d4e6-79b4-41b8-a8e8-8936c66c0577-must-gather-output\") pod \"must-gather-bnv4d\" (UID: \"74e4d4e6-79b4-41b8-a8e8-8936c66c0577\") " pod="openshift-must-gather-dprpk/must-gather-bnv4d" Jul 15 21:45:57.436499 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.436361 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxpgv\" (UniqueName: \"kubernetes.io/projected/74e4d4e6-79b4-41b8-a8e8-8936c66c0577-kube-api-access-wxpgv\") pod \"must-gather-bnv4d\" (UID: \"74e4d4e6-79b4-41b8-a8e8-8936c66c0577\") " pod="openshift-must-gather-dprpk/must-gather-bnv4d" Jul 15 21:45:57.537468 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.537438 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxpgv\" (UniqueName: \"kubernetes.io/projected/74e4d4e6-79b4-41b8-a8e8-8936c66c0577-kube-api-access-wxpgv\") pod \"must-gather-bnv4d\" (UID: \"74e4d4e6-79b4-41b8-a8e8-8936c66c0577\") " pod="openshift-must-gather-dprpk/must-gather-bnv4d" Jul 15 21:45:57.537655 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.537521 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/74e4d4e6-79b4-41b8-a8e8-8936c66c0577-must-gather-output\") pod \"must-gather-bnv4d\" (UID: \"74e4d4e6-79b4-41b8-a8e8-8936c66c0577\") " pod="openshift-must-gather-dprpk/must-gather-bnv4d" Jul 15 21:45:57.537847 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.537825 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/74e4d4e6-79b4-41b8-a8e8-8936c66c0577-must-gather-output\") pod \"must-gather-bnv4d\" (UID: \"74e4d4e6-79b4-41b8-a8e8-8936c66c0577\") " pod="openshift-must-gather-dprpk/must-gather-bnv4d" Jul 15 21:45:57.546940 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.546916 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxpgv\" (UniqueName: \"kubernetes.io/projected/74e4d4e6-79b4-41b8-a8e8-8936c66c0577-kube-api-access-wxpgv\") pod \"must-gather-bnv4d\" (UID: \"74e4d4e6-79b4-41b8-a8e8-8936c66c0577\") " pod="openshift-must-gather-dprpk/must-gather-bnv4d" Jul 15 21:45:57.637219 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.637126 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dprpk/must-gather-bnv4d" Jul 15 21:45:57.784491 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.784464 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dprpk/must-gather-bnv4d"] Jul 15 21:45:57.785994 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:45:57.785968 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74e4d4e6_79b4_41b8_a8e8_8936c66c0577.slice/crio-7be1063547a51f7abe93a02f79197e37a37af21c1c85bc86d880469effebf8e5 WatchSource:0}: Error finding container 7be1063547a51f7abe93a02f79197e37a37af21c1c85bc86d880469effebf8e5: Status 404 returned error can't find the container with id 7be1063547a51f7abe93a02f79197e37a37af21c1c85bc86d880469effebf8e5 Jul 15 21:45:57.824288 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:57.824261 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dprpk/must-gather-bnv4d" event={"ID":"74e4d4e6-79b4-41b8-a8e8-8936c66c0577","Type":"ContainerStarted","Data":"7be1063547a51f7abe93a02f79197e37a37af21c1c85bc86d880469effebf8e5"} Jul 15 21:45:59.842459 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:59.842406 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dprpk/must-gather-bnv4d" event={"ID":"74e4d4e6-79b4-41b8-a8e8-8936c66c0577","Type":"ContainerStarted","Data":"ec0aae84b32e51ef67d35468a41d23540b71b1a3ed9810c2cc322299f2e1de7b"} Jul 15 21:45:59.843026 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:59.842991 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dprpk/must-gather-bnv4d" event={"ID":"74e4d4e6-79b4-41b8-a8e8-8936c66c0577","Type":"ContainerStarted","Data":"4bdd3aa6f1f3058658dda5a6af5bc36a7d2c1d4aa85654e36f54045cdcac01e8"} Jul 15 21:45:59.865135 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:45:59.864473 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dprpk/must-gather-bnv4d" podStartSLOduration=1.9061752749999998 podStartE2EDuration="2.864451932s" totalImagesPullingTime="958.276657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 21:45:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:45:59.863391728 +0000 UTC m=+2812.953624905" watchObservedRunningTime="2026-07-15 21:45:59.864451932 +0000 UTC m=+2812.954685111" Jul 15 21:46:00.405294 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:00.405265 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-98h25_01016934-00c3-4dae-8db6-a118e7c3cbe9/global-pull-secret-syncer/0.log" Jul 15 21:46:00.660690 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:00.660616 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zjhlw_de9654f9-9400-474b-8978-582043f3d11e/konnectivity-agent/0.log" Jul 15 21:46:00.735768 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:00.735739 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-80.ec2.internal_3eb8d182e636c274e1ac9b37ef4dec65/haproxy/0.log" Jul 15 21:46:05.026893 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.026859 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz_1fe444cd-bc64-49e6-a643-f175f8840a7c/extract/0.log" Jul 15 21:46:05.053752 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.053670 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz_1fe444cd-bc64-49e6-a643-f175f8840a7c/util/0.log" Jul 15 21:46:05.080249 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.080216 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vfsbz_1fe444cd-bc64-49e6-a643-f175f8840a7c/pull/0.log" Jul 15 21:46:05.146254 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.146209 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q_9450047d-2a40-4ea7-a887-9dbe4c48269d/extract/0.log" Jul 15 21:46:05.178238 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.178202 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q_9450047d-2a40-4ea7-a887-9dbe4c48269d/util/0.log" Jul 15 21:46:05.210528 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.210490 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9w5q_9450047d-2a40-4ea7-a887-9dbe4c48269d/pull/0.log" Jul 15 21:46:05.241727 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.241691 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9_1348b943-daec-459c-9b72-cd854faa5e8b/extract/0.log" Jul 15 21:46:05.268627 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.268600 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9_1348b943-daec-459c-9b72-cd854faa5e8b/util/0.log" Jul 15 21:46:05.293690 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.293666 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wtrw9_1348b943-daec-459c-9b72-cd854faa5e8b/pull/0.log" Jul 15 21:46:05.332271 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.332180 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq_97209923-879b-4976-a942-b09d8672b61f/extract/0.log" Jul 15 21:46:05.360259 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.360233 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq_97209923-879b-4976-a942-b09d8672b61f/util/0.log" Jul 15 21:46:05.387775 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:05.387734 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18gfpq_97209923-879b-4976-a942-b09d8672b61f/pull/0.log" Jul 15 21:46:06.097819 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:06.097775 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-89h59_01f63185-f7d1-4781-b61a-58705dc8439b/manager/0.log" Jul 15 21:46:06.195435 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:06.195385 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-56hfv_c86f5dcf-93c5-428a-90f7-3f7f710b3d5b/registry-server/0.log" Jul 15 21:46:06.422956 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:06.422872 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jwdsh_64f52d57-004a-4fee-a8f1-36e939ccab01/manager/0.log" Jul 15 21:46:08.281361 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.281327 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7/alertmanager/0.log" Jul 15 21:46:08.332123 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.332092 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7/config-reloader/0.log" Jul 15 21:46:08.360066 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.360004 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7/kube-rbac-proxy-web/0.log" Jul 15 21:46:08.396257 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.396212 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7/kube-rbac-proxy/0.log" Jul 15 21:46:08.449192 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.448947 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7/kube-rbac-proxy-metric/0.log" Jul 15 21:46:08.495065 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.494978 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7/prom-label-proxy/0.log" Jul 15 21:46:08.526960 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.526927 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d5c9d1f6-b33d-455c-97f7-ce5d5736a2d7/init-config-reloader/0.log" Jul 15 21:46:08.573066 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.573037 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-6pcv7_84e688bb-4303-4ac4-b180-3ae21fa17823/cluster-monitoring-operator/0.log" Jul 15 21:46:08.611450 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.611421 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-nl9x8_3a49d89f-adde-46c4-add4-968cc8e595ea/kube-state-metrics/0.log" Jul 15 21:46:08.632435 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.632405 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-nl9x8_3a49d89f-adde-46c4-add4-968cc8e595ea/kube-rbac-proxy-main/0.log" Jul 15 21:46:08.654943 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.654914 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-nl9x8_3a49d89f-adde-46c4-add4-968cc8e595ea/kube-rbac-proxy-self/0.log" Jul 15 21:46:08.685003 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.684828 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b6d894486-d76t9_cb912f3a-ca31-4e5a-ae84-905f9198d5bc/metrics-server/0.log" Jul 15 21:46:08.711085 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.711058 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-jlpb4_3b8d56aa-8bc0-4f5e-8acb-697b9a6b8cc9/monitoring-plugin/0.log" Jul 15 21:46:08.744077 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.744043 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4ltr5_1270fd96-fdce-4907-858d-8047350e1ee5/node-exporter/0.log" Jul 15 21:46:08.770718 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.770686 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4ltr5_1270fd96-fdce-4907-858d-8047350e1ee5/kube-rbac-proxy/0.log" Jul 15 21:46:08.798721 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:08.798692 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4ltr5_1270fd96-fdce-4907-858d-8047350e1ee5/init-textfile/0.log" Jul 15 21:46:09.025230 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:09.025138 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mcpn5_8422f2da-72a1-4ad1-af11-7b9bcd44db03/kube-rbac-proxy-main/0.log" Jul 15 21:46:09.049583 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:09.049528 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mcpn5_8422f2da-72a1-4ad1-af11-7b9bcd44db03/kube-rbac-proxy-self/0.log" Jul 15 21:46:09.083191 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:09.083130 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mcpn5_8422f2da-72a1-4ad1-af11-7b9bcd44db03/openshift-state-metrics/0.log" Jul 15 21:46:10.250757 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.250725 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj"] Jul 15 21:46:10.256345 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.256316 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.266653 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.266617 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj"] Jul 15 21:46:10.409188 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.409128 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdb6p\" (UniqueName: \"kubernetes.io/projected/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-kube-api-access-tdb6p\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.409379 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.409311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-podres\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.409379 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.409348 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-lib-modules\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.409504 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.409398 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-proc\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.409504 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.409427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-sys\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.510520 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.510486 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-podres\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.510730 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.510528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-lib-modules\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.510730 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.510573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-proc\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.510730 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.510595 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-sys\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.510730 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.510631 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdb6p\" (UniqueName: \"kubernetes.io/projected/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-kube-api-access-tdb6p\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.510730 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.510692 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-proc\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.510730 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.510692 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-podres\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.511011 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.510739 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-sys\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.511011 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.510881 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-lib-modules\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.519830 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.519798 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdb6p\" (UniqueName: \"kubernetes.io/projected/0c6cae29-9ef6-481d-90e3-203d8dcc27cb-kube-api-access-tdb6p\") pod \"perf-node-gather-daemonset-6v9sj\" (UID: \"0c6cae29-9ef6-481d-90e3-203d8dcc27cb\") " pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.571130 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.571090 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.764199 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.763941 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj"] Jul 15 21:46:10.767703 ip-10-0-135-80 kubenswrapper[2573]: W0715 21:46:10.767640 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c6cae29_9ef6_481d_90e3_203d8dcc27cb.slice/crio-bfaea6da5dfba7080a7c8e86a84b690173940ea674eba07959a68ba7415b1818 WatchSource:0}: Error finding container bfaea6da5dfba7080a7c8e86a84b690173940ea674eba07959a68ba7415b1818: Status 404 returned error can't find the container with id bfaea6da5dfba7080a7c8e86a84b690173940ea674eba07959a68ba7415b1818 Jul 15 21:46:10.953379 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.953336 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" event={"ID":"0c6cae29-9ef6-481d-90e3-203d8dcc27cb","Type":"ContainerStarted","Data":"45736733a30a2364f956d0bd39283ec464ae6ecff7c2c98ef4a45f5010df8837"} Jul 15 21:46:10.953558 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.953435 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" event={"ID":"0c6cae29-9ef6-481d-90e3-203d8dcc27cb","Type":"ContainerStarted","Data":"bfaea6da5dfba7080a7c8e86a84b690173940ea674eba07959a68ba7415b1818"} Jul 15 21:46:10.953674 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.953656 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:10.974819 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:10.974759 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" podStartSLOduration=0.974740897 podStartE2EDuration="974.740897ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 21:46:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 21:46:10.970056226 +0000 UTC m=+2824.060289404" watchObservedRunningTime="2026-07-15 21:46:10.974740897 +0000 UTC m=+2824.064974076" Jul 15 21:46:11.434901 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:11.434867 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/1.log" Jul 15 21:46:11.443080 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:11.443054 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-jwz4p_cb05969b-c83d-4c37-8c63-1221309d2f8b/console-operator/2.log" Jul 15 21:46:12.578683 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:12.578657 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-rs6fp_79083cd2-3480-42d0-bdf5-a5719b9f8995/volume-data-source-validator/0.log" Jul 15 21:46:13.651697 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:13.651670 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jmfmx_47f9f4d6-0555-4123-b64b-5f753c9e1160/dns/0.log" Jul 15 21:46:13.673445 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:13.673419 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jmfmx_47f9f4d6-0555-4123-b64b-5f753c9e1160/kube-rbac-proxy/0.log" Jul 15 21:46:13.805272 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:13.805249 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pw8ln_17f76060-64b0-4ec1-ae6e-18eaea915761/dns-node-resolver/0.log" Jul 15 21:46:14.389340 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:14.389308 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6pkk7_fda8f067-5b21-4d25-9771-77f72ab05e60/node-ca/0.log" Jul 15 21:46:15.398883 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:15.398856 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46vn9s_36b9ccbb-9fa2-4561-b234-49b301aeee60/istio-proxy/0.log" Jul 15 21:46:15.691815 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:15.691740 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qf9dp_cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7/istio-proxy/0.log" Jul 15 21:46:15.735603 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:15.735571 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qf9dp_cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7/istio-proxy/1.log" Jul 15 21:46:15.814687 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:15.814659 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-l6hpz_8cf3a567-8ba9-41a3-8b71-9296626ccf35/payload-processing/0.log" Jul 15 21:46:15.836933 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:15.836904 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585bb7b99d-xcp26_14cd8b87-f21c-41c3-8cb8-7893d3f5fa28/router/0.log" Jul 15 21:46:16.557083 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:16.557043 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5jlnp_4ec81d32-4e52-4f6e-a9a9-a4c0d19c0ef1/serve-healthcheck-canary/0.log" Jul 15 21:46:16.972722 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:16.972644 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dprpk/perf-node-gather-daemonset-6v9sj" Jul 15 21:46:17.100645 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:17.100609 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-xsxss_3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88/insights-operator/0.log" Jul 15 21:46:17.101470 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:17.101449 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-xsxss_3c460d9e-76ff-4e7f-8d04-f90f2b8c2b88/insights-operator/1.log" Jul 15 21:46:17.227040 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:17.226957 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r8j48_9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d/kube-rbac-proxy/0.log" Jul 15 21:46:17.251383 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:17.251357 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r8j48_9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d/exporter/0.log" Jul 15 21:46:17.278845 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:17.278822 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r8j48_9eb747fc-0ae3-4839-b7f7-11c0a84d3d9d/extractor/0.log" Jul 15 21:46:20.252772 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:20.252714 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-59d5c68977-fhdzm_c3ca1567-0a3b-49e8-b166-c368e2d87553/manager/0.log" Jul 15 21:46:20.278076 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:20.277984 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-ccf85_85601b0c-aa09-4e72-a515-784c4d9661ba/server/0.log" Jul 15 21:46:20.559725 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:20.559698 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46vn9s_36b9ccbb-9fa2-4561-b234-49b301aeee60/istio-proxy/0.log" Jul 15 21:46:20.608648 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:20.608619 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74776f8bd9-xm2jb_7c6d6487-2ef6-44c1-b438-cc3c92ad3797/manager/0.log" Jul 15 21:46:20.631191 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:20.631141 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74776f8bd9-xm2jb_7c6d6487-2ef6-44c1-b438-cc3c92ad3797/copy-manifests/0.log" Jul 15 21:46:20.820576 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:20.820478 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qf9dp_cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7/istio-proxy/0.log" Jul 15 21:46:20.860041 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:20.860009 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qf9dp_cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7/istio-proxy/1.log" Jul 15 21:46:20.916782 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:20.916755 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-l6hpz_8cf3a567-8ba9-41a3-8b71-9296626ccf35/payload-processing/0.log" Jul 15 21:46:20.931775 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:20.931745 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585bb7b99d-xcp26_14cd8b87-f21c-41c3-8cb8-7893d3f5fa28/router/0.log" Jul 15 21:46:21.911795 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:21.911761 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-649f4f97b8-qpm5c_099f541c-62a5-4682-b722-b519406ed95e/manager/0.log" Jul 15 21:46:21.945530 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:21.945501 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-g7hn4_52150eab-8243-47dd-8b9d-283896f16a8e/openshift-lws-operator/0.log" Jul 15 21:46:22.196270 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:22.196228 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46vn9s_36b9ccbb-9fa2-4561-b234-49b301aeee60/istio-proxy/0.log" Jul 15 21:46:22.455211 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:22.455106 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qf9dp_cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7/istio-proxy/0.log" Jul 15 21:46:22.499605 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:22.499580 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-qf9dp_cf0e75e7-6f3a-42b2-9069-6b6f8bd7d4c7/istio-proxy/1.log" Jul 15 21:46:22.556165 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:22.556125 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-l6hpz_8cf3a567-8ba9-41a3-8b71-9296626ccf35/payload-processing/0.log" Jul 15 21:46:22.570539 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:22.570488 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-585bb7b99d-xcp26_14cd8b87-f21c-41c3-8cb8-7893d3f5fa28/router/0.log" Jul 15 21:46:27.264070 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:27.264043 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-rvmvc_a35d610b-d08f-47d4-99c9-9f6aa55fa6ff/kube-storage-version-migrator-operator/1.log" Jul 15 21:46:27.264949 ip-10-0-135-80 kubenswrapper[2573]: I0715 21:46:27.264929 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-rvmvc_a35d610b-d08f-47d4-99c9-9f6aa55fa6ff/kube-storage-version-migrator-operator/0.log"